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Dear  Reader, 

On  behalf  of  the  distinguished  members  of  the  Joint  Directors  of 
Laboratories  Data  Fusion  Subpanel  (JDL/DFS),  session  chairmen  and  authors,  it 
is  indeed  a  pleasure  for  me  to  present  to  you  the  Technical  Proceedings  of  the 
1988  Tri-Service  Data  Fusion  Symposium.  The  technical  strength  and  uniqueness 
of  this  conference  and  these  proceedings  are  directly  attributable  to  the  hard 
work  and  careful  considerations  of  many  talented  people.  The  quality  of  the 
program  sessions,  technical  paper  content,  and  invited  presentations  is  due 
entirely  to  their  efforts.  In  structuring  the  conference  we  tried  to  create  a 
program  that  was  relevant,  current,  and  functionally  interrelated  which 
encompassed  the  needs  of  the  user,  the  researcher,  the  designer,  and  the 
developer.  Accordingly,  the  papers  covered  the  gamut  from  theory  to 
application. 

Our  conference  success,  measured  by  the  high  quality  of  the  papers,  the 
caliber  of  the  invited  senior  military  speakers,  the  national  and  inter¬ 
national  prominence  of  our  authors,  presenters,  and  session  chairmen  combined 
with  the  overwhelming  registration  from  all  sectors  of  the  technical  community 
nationwide,  has  repaid  us  for  all  the  time,  effort,  and  concern  exhibited 
during  the  conception  and  planning  phases.  I  thank  all  of  you  sincerely  for 
your  early  confidence,  for  your  continued  support,  and  for  the  very  kind  and 
laudatory  comments  we  received.  The  JDL/DFS  is  already  planning  the  DFS-89 
agenda  utilizing  DFS-88  lessons  learned  plus  fresh  ideas,  concepts,  and 
imagination. 

Thanks  again  for  your  continued  interest  and  support.  I  hope  this 
document  provides  for  you  a  useful,  lasting  reference  and  momento  of  DFS-88. 


Otto  Kessler 
Coordinator,  DFS-88 
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FOREWORD 


The  1988  Tri-Serv1ce  Data  Fusion  Symposium  (DFS-88)  was  held  at  Laurel, 
Maryland  on  17-19  May  1988  under  the  joint  sponsorship  of  the  Data  Fusion 
Sub-Panel  of  the  Joint  Directors  of  Laboratories  (JDL/DFSP),  CECOM  Center  for 
Signals  Warfare,  Naval  Air  Development  Center,  Rome  Air  Development  Center, 
and  the  Naval  Ocean  Systems  Center.  The  symposium  theme,  THE  ROLE  OF  DATA 
FUSION  IN  TACTICAL  C^,  emphasizes  (a)  the  emergence  of  technology  for 
implementing  data  fusion  in  military  systems,  and  (b)  the  growing  recognition 
of  the  role  of  data  fusion  in  the  data  collection/information  generation 
processes  required  for  situation  awareness  and  threat  assessment  for  both 
tactical  and  strategic  purposes. 

The  purpose  of  the  symposium  was  to  provide  a  professional  forum  for  the 
broad  exchange  of  information  and  ideas  on  the  development  of  data  fusion 
technology  for  DOD  applications.  That  we  have  made  a  significant  first  step 
in  this  direction  is  attested  to  by  the  quality  of  the  technical  papers  and 
the  level  of  interest  evidenced  by  the  large  number  of  attendees  drawn  from  a 
broad  spectrum  of  disciplines.  The  symposium: 

0  attracted  468  registered  attendees  (303  from  industry  and  165  from 
government  [military/civilian] ) ; 

0  included  operational  C^  briefs  by  Army,  Navy,  Air  Force  and  Marine 
Corps  representatives; 

0  featured  36  technical  papers  dealing  with  research,  development,  and 
system  applications  of  data  fusion  technologies  and  techniques;  and 

0  a  commander's  tactical  C^  perspective  forum  relating  to  multi-service 
experience  involving  command  and  control  functions,  information 
handling,  and  information  fusion. 

In  publishing  these  Proceedings  the  sponsoring  activities  seek  to 
facilitate  an  effective  and  timely  dissemination  of  technical  information. 

The  Proceedings  comprise  two  volumes.  Volume  I  contains  the  unclassified 
papers  available  at  the  time  of  the  symposium.  Volume  II  contains  the 
classified  papers  and  unclassified  papers  received  late.  The  papers  contained 
in  this  document  were  printed  directly  from  unedited  reproducible  copies 
submitted  by  the  authors  who  are  solely  responsible  for  their  contents. 
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BEKAA  VALLEY  CAMPAIGN  1982 
Maj  Gen  Doyle  E.  Larson,  USAF  (Ret) 

I  want  to  talk  to  you  today  about  the  Bekaa  Valley  Campaign 
of  1982  from  the  viewpoint  of  the  Combatant  Element  Commander.  The 
p0rspective  is  from  the  Corps  division  level  and  other  levels  in  my 
perception  of  what  happened  in  the  Bekaa  Valley.  At  the  time  this 
occurred,  almost  six  years  ago,  I  was  not  only  the  Commander  of  the 
Electronic  Security  Command,  but  I  was  the  Director  of  the  Joint  Electronic 
Warfare  Center.  It  was  my  responsibility  to  keep  the  Joint  Chiefs 

informed  on  all  things  "electronic"--that  is,  electronic  warfare,  electronic 
combat,  C^  countermeaures — etc.  After  the  incursion  into  Lebanon  I 

began  working  very  hard  with  the  National  Security  Agency  to  examine 
and  assess  the  data  that  we  had,  and  with  DIA  to  find  out  exactly 
what  had  happened  in  the  Bekaa  Valley.  Even  after  this  data  analysis, 
there  were  still  operational  issues  that  were  unanswered.  What  did 
the  Commander  attempt  to  do?  What  kind  of  tactics  did  he  use?  And 
so  I  sent  one  of  my  officers  over  to  Tel  Aviv  to  try  to  get  answers 

to  this  question  of  tactics  and  operations,  and  to  my  dismay  when 

he  returned  he  had  been  told  either  lies  or  nothing,  which  is  sort 
of  good  Israeli  OP  SEC.  As  a  result,  when  I  retired  in  1983,  I  had 
an  excellent  idea  from  an  intelligence  standpoint  what  had  happened 
in  the  Bekaa  Valley,  but  the  operator  view  was  still  missing.  Fortunately 
I  was  invited  over  to  Israel,  spent  ten  days  with  the  Israeli  Defense 
Forces  up  into  the  Bekaa  Valley  for  a  day  and  a  half,  and  down  into 

the  Sinai  where  I  had  the  good  fortune  to  run  into  the  Chief  of  Israeli 

Intelligence  Production  during  the  incursion,  and  had  a  chance  to  talk 
about  the  situation.  When  it  kicked  off  on  9  June  1982  it  had  culminated 
several  years  of  planning.  I  became  aware  of  that  because  of  the 

fact  that  the  OP's  order.  Peace  For  Galilee,  was  signed  by  Ezer  Weitzman. 

He  hadn’t  been  the  Minister  of  Defense  for  over  two  years  when  the 

incursion  took  place  which  told  me  immediately  that  the  Israeli  Air 
Force  had  that  plan.  Peace  For  Galilee,  in  their  hands  for  over  two 

years  and  I’m  sure  had  been  practicing  and  drilling  for  the  execution. 
And  so  when  the  day  finally  came  when  the  Israeli  government  had  had 
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all  they  could  take  from  the  PLO  and  they  determined  they  were  going 
to  do  something  about  it,  they  issued  the  order  to  Maj  Gen  David  Evri , 
the  Commander  of  the  Israeli  Air  Force,  to  begin  execution  of  Peace 
For  Galilee.  So  he  drove  up  the  road  to  his  command  post  at  Mt .  Maron 
and  began  executing  a  plan  that  he  had  been  working  on  for  a  long 
t  ime . 


Now  going  back  at  least  two  years  then,  the  first  thing  David 
Evri  recognized  was  that  to  drive  the  PLO  out,  destroy  their  base, 
and  defeat  the  Syrian  forces,  you  are  going  to  have  to  move  some  columns 
of  Armor  infantry  North,  and  he  was  going  to  have  to  provide  air  support 
for  that  operation.  And  as  he  thought  about  that,  David  Evri  said, 
"I  am  not  going  to  make  the  mistake  that  we  made  in  Yom  Kippur — before 
we  go  after  tanks  and  artillery  and  troops,  we  are  going  to  suppress 
the  air  defense  and  we  are  going  to  gain  air  supriority."  You  recall 
in  Yom  Kippur  in  the  first  five  days  they  lost  one-third  of  their 

Air  Force.  And  so,  David  Evri  said,  we’re  not  going  to  do  that  in 
Peace  For  Galilee.  We  are  going  to  go  in  there  and  get  control  of 

that  air  space  first  and  foremost  and  then  we’ll  support  tanks.  And 
so  he  met  with  his  staff  and  he  informed  his  staff  that  he  wanted 
to  exercise  what  he  called  command  and  control  warfare.  He  was  going 
to  attack  the  Syrian’s  ability  to  control  their  forces.  That  was 

his  game  plan  and  he  wanted  to  use  a  disruptive  strategy,  which  to 

David  Evri  meant  he  wanted  to  be  able  to  combine  deception  and  jamming 
with  his  lethal  capability.  He  identified,  essentially,  four  tools 
that  he  intended  to  use  in  that  incursion.  He  wanted  to  use  his  lethal 
capability,  of  course,  but  he  also  wanted  to  jam  and  use  some  deception 
in  a  number  of  ways.  However,  the  critical  part  of  this  strategy, 
the  tool  that  is  most  important,  and  which  must  be  in  place  at  the 
outset,  is  what  he  called  intelligence  exploitation.  But  he  said  I 
want  to  exploit  the  enemy’s  command  and  control  systems  and  identify 
the  weaknesses  that  I  can  exploit.  And  so  he  met  with  his  intelligence 
people  and  they  began  to  show  General  Evri  what  was  in  the  database. 
They  told  him  that  they  had  a  pretty  good  handle  on  artillery,  self-propelled 
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artillery  in  particular,  and  major  elements  which  have  been  located. 
Their  database  on  infantry  was  pretty  complete — down  to  fairly  small 
sized  units — although  precise  locations  were  not  available.  They  also 

had  a  pretty  good  handle  on  armor  but  did  not  know  where  the  division 
bunker  was,  nor  their  headquarters.  And  then,  of  course,  Evri  had 
mentioned  air  defense  as  a  number  one  consideration.  And  they  were 

very  proud  to  tell  General  Evri  that  each  week  they  surveyed  each 

prepared  site  and  each  SAM  site  with  imagery  so  that  they  had  each 

site  down  to  one  meter  in  accuracy  and  they  felt  very  good  about  their 
locational  information  and  so  they  ended  up  very  proudly  showing  him 
what  was  in  the  Bekaa  Valley. 

General  Evri  said,  "Well,  that's  very  nice  guys,  but  that  isn't 

going  to  support  my  strategy."  Furthermore  he  said,  "I  don't  have 

enough  bombs  and  bullets  to  take  care  of  all  the  combatant  elements 

and  I  certainly  can't  jam  everything.  He  said,  "I  want  you  to  go 

back  to  the  drawing  board,  and  I  want  you  to  do  a  nodal  analysis. 

I  want  nodal  intelligence."  They  came  back  eventually  and  they  had 
the  nodes.  Between  each  one  of  the  nodes  for  armor  they  identified 

the  type  of  communication  link  hooking  those  nodes  together.  They 

showed  this  to  General  Evri  and  as  a  matter  of  great  pride  they  said 

in  the  process  of  doing  this  nodal  analysis  they  ended  up  putting 
imagery  and  SIGINT  together  (fusing  it)  and  found  the  bunker,  ^nd  it 
was  only  when  we  did  that  that  they  could  find  that  bunker.  General 

Evri  was  then  interested  in  the  red  line  going  down  to  the  tank  maneuver 

elements  and  he  said  "What  is  that  red  line?"  They  said  that's  a 

Soviet  built  low  VH  radio.  He  said,  "You  mean  to  tell  me  that's  the 

only  way  they  communicate  at  that  level?"  And  they  said,  "Yes,  it 

is."  And  they  said,  here's  the  surface-to-air  missile  and  Triple  A 
problem  and  again  the  nodes  were  explained.  Once  again  the  single 
red  line  caught  David  Evri's  eye  and  he  said,  "You  mean  to  tell  me 

that  the  launcher  receives  tracking  data  from  higher  echelons  and  receives 
its  command  and  control  only  on  that  low  VH  line?"  And  they  said, 

"Yes."  And  then  he  said,  "I  assume  that's  an  encrypted  line?"  And 
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they  said,  "No,  it's  unencrypted."  He  said,  "Well  that 's  very  interesting." 
"What's  the  frequency  range?"  They  said  between  30  and  50  MHz.  Now 

that  was  the  same  Bekaa  Valley  when  they  got  through  this  nodal  analysis. 
Same  number  of  combatant  elements,  only  now  David  Evri  was  only  looking 
at  a  hand  full  of  nodes  and  a  single  red  line. 

Then  David  Evri  said  something  else  to  his  Intelligence  people. 
He  said,  "Every  sensor  that  you  manage  has  to  do  two  jobs  simultaneously." 
On  the  right  hand  side  of  the  chart  Evri  said  I  want  you  to  correlate 
and  fuse,  validate  and  analyze,  and  give  me  the  best  all-source  data 

you  can  each  and  every  day  up  until  the  battle  starts  and  I  am  going 

to  use  this  for  my  major  planning  and  for  targeting  of  fixed  facilities. 
And  certainly  at  the  end  of  each  day's  battle,  1  want  to  have  battle 
damage  assessment  so  that  I  know  what  I  need  to  adjust  based  upon 

every  source  you've  got  available  before  I  start  tomorrow’s  battle." 

But  he  said,  "When  the  balloon  goes  up,  each  sensor  you're  operating 

has  also  got  to  deliver  the  left  side  of  the  chart  for  me  also,  that 
is,  essentially  locational  data.  I  want  you  to  tell  me  what  is  it, 

where  is  it,  and  when  did  you  detect  it,  and  I  want  it  as  fast  as 

you  can  give  it  to  me.  And  you  can  tell  me  how  inaccurate  it  is  if 
you  want  to  but  I  have  to  have  it  fast.  If  you  can't  meet  the  speed 
requirement,  give  it  to  me  at  the  end  of  the  day  in  the  all-source, 

fused,  analyzed  form.  But  in  order  to  use  information  for  real-time 
targeting  and  to  chase  mobile  things,  things  that  scoot  and  shoot  on 

the  field  of  the  Bekaa  Valley,  I  want  in  some  cases  single  sensor 

data  as  fast  as  you  can  get  it  to  me.  And  I  will  live  with  the  inaccuracies 
if  you  will  just  tell  me  what  they  are.  Let  me  make  judgments  on 

how  I  can  use  the  data. 

The  day  came  when  Evri  was  told  to  execute  Peace  For  Galilee 

and  so  he  began  launching  his  intelligence  assets.  Some  of  my  slides 
now  were  not  taken  in  Israel.  Some  of  them  show  U.S.  comparable  equipment. 
The  U.S.  did  in  fact  build,  E-Systems,  Greenville,  built  an  excellent 
ELINT  aircraft  with  a  data  link  right  into  the  Mt.  Maron  command  post. 


4 


It  was  an  important  asset.  Evri  put  it  in  place  early  on  that  day. 
ESL  has  built  them  a  GUARDRAIL  system;  it  does  an  excellent  job.  The 

receivers  are  controlled  on  the  ground.  An  again  the  data  was  on 

the  ground  and  was  available  to  Evri  directly.  Probably  the  most 
important  intelligence  support  for  General  Evri,  however,  was  on  top 
of  Mt.  Hermon  where  since  the  1967  war  the  Israelis  have  occupied 

the  high  ground  and  have  some  excellent  COMINT  and  ELINT  equipment 
up  there.  Looking  to  the  east  you  can  see  all  the  Syrian  airfields. 
Line  of  sight  coverage  is  very  good  and  looking  to  the  west  you  can 
see  all  the  way  down  to  downtown  Beruit.  So  he  had  excellent  line 

of  sight  coverage.  The  Mt.  Hermon  command  bunker  is  located  just 
below  the  "e"  in  Israel  on  the  slide.  When  you  drive  past  it,  it 

is  not  a  very  prominent  peak,  but  it  is  high  enough  so  that  Evri  had 
connectivity  with  his  intelligence  sensors  and  he  had  connectivity  with 
his  jammers  and  with  the  rest  of  his  command  and  control  assets.  I 

asked  permission  each  day  of  the  ten  days  I  was  there  to  visit  Mt. 

Hermon  and  each  morning  they  would  sound  very  promising  that  they  would 
take  me  there,  but  I  never  got  there.  So  I  have  to  imagine  what  it 
looks  like  inside.  I  know  that  they  were  receiving  a  lot  of  voice 
reports.  I  know  they  were  receiving  some  digital  data  and  display, 
but  I  suspect  they  had  a  combination  of  very  fast  grease  pencil  displays 
and  some  digital  displays  and  the  voice  mostly  coming  down  from  Mt. 
Hermon  to  Mt .  Maron  provided  the  basis  for  what  was  being  plotted 
here.  When  he  had  his  intelligence  sensors  up  and  looking  at  those 
things  that  they  knew  he  wanted  to  look  at  first,  he  began  launching 

his  aircraft. 

Here  is  a  case  where  he  decided  to  use  some  deception.  He 
was  obviously  going  to  launch  every  aircraft  silently —  "IFF,  no  radar, 
no  radio  communications,  launch  with  flares  only."  But  just  to  make 
sure  that  the  U.S.  RC135  and  the  P-3s  operating  out  over  the  Med  didn’t 
detect  a  rather  large  size  operation  underway,  he  also  launched  a  bunch 
of  training  missions  using  operational  IFF  codes  and  call  signs.  And 
to  our  linguists  onboard  the  RC135  that  morning,  the  morning  seemed 
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fairly  routine.  The  2A7  aircraft  got  in  place  and  began  low-level 
orbiting  and  with  the  time  set  up  just  right,  which  had  been  practiced 
in  the  Sinai,  at  the  precise  minute  Evri  launched  some  special  forces 
to  go  after  the  SAM  sites  in  the  Bekaa  Valley.  When  the  timing  was 
even  a  little  more  close  to  the  actual  kick  off  of  the  combat,  he 
then  began  launching  some  artillery  rounds  at  the  southern  most  Syrian 
units.  So  he  was  getting  pretty  good  suppression  of  air  defense  from 
his  artillery.  And  then  finally  Evri  began  launching  some  minidrones. 
This  Melpar  drone  looks  an  awful  lot  like  the  Teddirane  drone  that 
I  watched  being  manufactured  in  the  factory  in  Tel  Aviv.  Evri  had 
a  couple  of  purposes  for  these  drones.  I  would  say  he  had  three  reasons 
to  use  drones.  The  first  was,  he  had  some  excellent  ELINT  equipment, 
but  you  can*t  collect  ELINT  if  the  enemy  doesn’t  emit.  And  so  he 
wanted  to  provoke  the  enemy  into  using  his  radar.  So  one  of  the  reasons 
for  having  drones  was  to  get  those  emitters  up  on  the  air  so  he  could 
confirm  their  location  and  do  something  with  it.  He  used  the  drones 
also  for  deception.  They  put  radar  repeaters  in  the  drones  which 
made  them  look  just  like  K-4’s  and  F-4s.  And  then  he  had  a  third 
purpose  for  the  drones.  And  that  was  to  carry  video  electro-optical 
and  capability  which  was  controllable  from  his  command  post.  This 
shows  you  the  actual  control  position  in  the  Mt.  Maron  bunker  with 
a  map  on  the  right  hand  side  showing  the  current  location  of  the  drone 
and  on  the  left  hand  side  the  controls  and  what  is  being  seen  by  the 
camera.  This  permitted  Evri  to  see  in  real-time  and  match  or  fuse 
imagery  with  his  SIGINT  coming  into  him  from  multiple  sources.  Excellent, 
all-source  fused  information. 

It  was  about  that  time  that  the  linguist  on  top  of  Mt.  Hermon 
began  responding  to  David  Evri’s  specific  guidance.  He  said  I  want 
you  to  look  at  the  SAMs  first  and  when  you  hear  them  controlling  their 
launchers,  I  want  you  to  repeat  verbatim  what  you  hear.  I  don’t  want 
you  to  analyze  it  or  summarize  it.  I  want  you  to  give  it  to  me  as 
you  get  it  by  voice  reports.  And  so  what  the  linguist  began  to  hear 
on  those  unencrypted  links  were  the  Syrian’s  taking  the  deception  bait. 
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They  saw  the  drones  coining  in  from  the  south  and  they  began  chattering 
at  their  launch  operators  that  In  fact  the  Israelis  were  beginning 
their  attack*  The  K-4s  and  F-4s  were  coming  in  from  the  south  and 
they  issued  firing  instructions.  And  as  soon  as  Evri  heard  those 
instructions  being  given  he  knew  they  had  taken  the  bait  so  he  wanted 
to  make  damn  sure  they  didn’t  get  a  countermanded  order  and  so  he 
turned  on  the  jammers.  Between  20  and  50  MHz  onboard  the  RC135,  the 
panoramic  display  scope  looked  milky  white  with  noise.  No  launch  operator 
was  going  to  get  anymore  information  from  up  channel  nor  was  he  going 
to  get  any  other  tracking  data.  He  was  pretty  much  on  his  own.  He 
was  going  to  have  to  use  his  own  radar  and  his  own  optical  tracker. 

At  that  point  Evri  issued  one  code  word — ATTACK  to  his  237  aircraft 
and  at  3  o’clock  in  the  afternoon  at  4,000  to  6,000  ft.  the  attack 
started.  If  you  look  toward  the  west  from  Bekaa  Valley  at  3  o’clock 
in  the  afternoon  I  guarantee  you  will  see  a  very  bright  sun  and  your 
optical  tracker  will  be  absolutely  useless  and  that’s  exactly  what 
happened.  In  ten  minutes  they  destroyed  every  SAM  site  in  the  Bekaa 
Valley.  I  talked  to  a  lot  of  the  pilots  who  were  involved  in  the 
operation.  I  looked  at  a  lot  of  the  film.  It  looked  a  lot  to  me 
like  they  used  dumb  bombs.  I  saw  neat  holes  in  every  control  van, 
every  launcher.  I  could  see  no  missiles  on  any  rails.  In  fact  the 

pilots  reported  not  seeing  any  missiles.  It  appears  that  they  had 
pickled  off  everything  they  had  against  the  drones  and  were  reloading 
when  they  caught  them.  As  soon  as  Evri  issued  that  code  word  for 

the  attack  to  start.  Col  Rahm,  his  Intelligence  Production  Chief,  knew 
immediately  that  Evri  was  no  longer  interested  in  SAMs.  He  had  done 

everything  required  to  destroy  those  SAM  sites.  David  Evri’s  information 
displays  were  changed  immediately  because  he  was  not  focusing  on  air 
threat,  Flogger  and  Migs.  And  that  code  word  to  start  the  attack 
was  a  signal  to  his  linquist  on  top  of  Mt.  Hermon  to  start  monitoring 
fighters  as  a  first  priority.  And  again  the  explicit  instructions 
that  he  had  given  those  linquists  was  the  same — report  exactly  what 
you  hear  as  you  hear  it.  I  want  it  verbatim.  What  they  began  to 
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hear  on  encrypted  VH  communication  links  were  the  controllers  talking 
to  the  lead  pilots  about  the  attack  going  on  in  the  Bekaa  Valley  and 
issuing  specific  guidance  to  those  Migs  and  Floggers.  What  happened 
was  that  the  senior  GCI  controller  made  a  very  grave  tactical  mistake. 
Out  over  the  Med  at  about  5,000  to  6,000  ft.  he  saw  another  wave  of 

aircraft  and  he  concluded  that  that  was  the  second  wave  of  ground 

attack  aricraft  and  so  he  advised  his  Floggers  and  Migs  to  execute 

the  blow-through  tactic.  This  is  the  same  tactic  the  Russians  taught 
the  Vietnamese.  Come  in  on  the  deck,  pull  up  sharply  through  the 
attacking  force  and  disrupt  their  bombing  attack. 

As  soon  as  Evri  heard  the  blow  through  command  being  given, 

the  jammer  started.  And  again  onboard  the  RC135  on  a  panoramic  display 
between  130  and  150  MHz  it  was  very  white.  If  you  were  flying  wing 
on  a  Flogger,  you  could  probably  talk  to  your  wingman.  But  you  weren’t 
going  to  talk  to  anybody  on  the  ground.  You  weren’t  going  to  talk 
to  anybody  more  than  a  few  miles  away.  So  essentially  the  last  instruction 
had  been  given  to  those  Floggers  and  Migs  and  sure  enough  into  the 

Bekaa  Valley  they  went  on  the  deck,  pulled  up  sharply  in  front  of 

the  combat  air  patrol.  F-16s  just  waiting  for  them.  In  ten  minutes 

they  shot  down  29  aircraft.  Down  in  the  Sinai  at  the  new  Ramone  Air 
Base  I  took  a  look  at  a  lot  of  the  F-16  film-gunner  film.  Every  film 

clip  looked  about  the  same.  Three  to  five  seconds  in  length.  The 

first  thing  you  would  see  on  the  F~16  heads  up  display  was  a  Flogger 
or  a  Mig  at  the  outside  edge.  You  would  see  the  wings  go  up  pretty 
sharply  and  you  would  see  an  AIM-9L  come  off  the  rail  and  splash  on 
each  Mig  or  Flogger.  In  fourteen  minutes  of  elapsed  time,  they  had 

destroyed  19  SAM  sites  and  shot  down  29  aircraft. 

For  all  intents  and  purposes,  you  could  say  David  Evri  had  air 

superiority.  He  went  on  to  shoot  down  a  total  of  80  aircraft,  lost 

one  F-4  when  the  pilot  got  a  little  too  slow  at  low  altitude  chasing 
a  ground  target  and  an  SA— 7  got  him.  Evri  manipulated  his  information. 
He  manipulated  those  tools  extremely  well.  Now  the  Joint  Chiefs  looked 
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at  that  and  declared  that  it  was  a  classic  case  of  countermeasures. 
Evri  glued  these  four  tools  together  with  OP  SEC.  He  was  smart  enough 
to  recognize  if  you  don't  have  OP  SEC,  you  don't  have  deception  except 
against  your  self.  He's  combat  experienced  enough  to  know  that  you 
don’t  have  effective  jamming  if  your  SIGINT  system  isn't  tethered  right 
to  your  jammers  because  you  are  going  to  be  jamming  the  wrong  things. 
You  are  not  going  to  be  effectively  jamming  and  so  you  are  just  going 
to  be  sending  electrons  out  in  the  air.  And  finally,  exploitation. 
Evri  controlled  the  sensors  for  that  battle.  He  did  not  go  to  Tel 
Aviv  and  appear  before  a  committee  to  plead  for  intelligence.  When 
he  said  I  want  you  to  listen  to  SAMs  now,  they  listed  to  SAMs.  When 
he  said  move  onto  airplanes,  they  moved  onto  airplanes.  When  he  said 
I  want  you  to  report  verbatim,  they  reported  verbatim.  Obviously  as 
you  go  up  in  complexity  in  command  arrangements  it  gets  to  be  more 
difficult . 

The  sad  state  of  the  truth  in  the  U.S.  is  that  today  for  the 
air-land  team  which  I  am  trying  to  depict  here — the  Army,  Air  Force, 
air-land  stream  or  for  that  matter  Navy,  Marine  power  projection,  we 
provide  damn  little  information  like  Evri  got  on  the  left  hand  side 
of  that  chart.  In  fact,  I  suggest  to  you  that  only  the  air  defense 

mission  for  the  Air  Force  gets  that  kind  of  support.  Air-to-mud  missions 

get  zip.  The  fighter  wings  and  the  divisions  get  close  to  zip.  Most 
of  the  stuff — I  was  just  in  Korea  for  Team  Spirit  and  the  three  star 

up  at  Camp  Red  Cloud  was  about  three  feet  off  the  ground  upset  with 

his  intelligence  support.  He  said  they  come  rushing  in  at  the  end 
of  the  battle  with  a  summary,  an  intelligence  summary;  this  report 

tells  me  how  I  lost,  he  says during  the  day  I  got  nothing.  And 

that  is,  generally  speaking,  the  state  of  affairs.  Here's  kind  of 
the  situation:  Our  intelligence  people  are  working  diligently  but 

maybe  we  have  too  many  managers.  Maybe  DIA  has  the  charter  but  not 
the  capability.  You  can't  transfer  our  intell  databases  easily,  and 
you  can't  access  them  easily.  And  there  certainly  aren't  any  nodal 
analyses.  There  is  an  effort  underway  in  San  Antonio  called  COMFY 
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WEB  which  started  out  as  a  genuine  effort  to  provide  a  database 
and  nodes*  It  is  now  so  unwieldly,  so  large  that  you  would  have  to 
carry  a  hugh  computer  to  the  field  with  you  to  do  anything  with  it* 
Furthermore,  the  SIGINT  system  can’t  feed  it  because  there  are  too 
many  data  elements  in  there.  In  point  of  fact,  I  think  it  is  absolutely 

excessive  to  have  more  than  the  19  data  elements  that  NATO  has  agreed 

to  exchange  in  that  database.  Speaking  for  the  commander  conducting 
command  and  control  warfare,  he  only  needs  to  know  what  is  it,  where 
is  it,  and  when  did  you  see  it.  And  he  can  find  out  frequency  information 
pretty  well  on  his  own.  Now  he  also  needs  to  get  some  stuff  fast. 
When  I  say  fast  obviously  intelligence  sensors  can’t  report  to  the 
commander  until  they’ve  got  resolved  data,  i.e.,  they  can’t  get  it 
until  they  capture  it.  But  once  they  do  capture  it  instead  of  typing 
up  a  paper  report,  it  would  be  very  nice  if  they  would  just  click 

out  a  symbol  indicating  the  location  of  that  threat  or  target  and 

then  type  up  the  paperwork.  What  happens  today  is  we  have  ’’Cadillac” 

collectors.  I  mean  the  finest  collection  systems  the  world  has  ever 

seen  operating  in  a  digital  world.  But  when  it  hits  the  ground  the 
information  largely  goes  into  record  communications.  It  becomes  a 
piece  of  paper  and  goes  out  of  the  digital  stream.  It  takes  time 

to  poke  up  messages.  It  takes  time  to  transmit  messages.  Generally 
speaking  in  Team  Spirit  the  air  commanders  were  getting  their  information 
in  27  minutes.  Up  to  Red  Cloud  it  was  two  hours.  It’s  tough  to  conduct 
armed  warfare  with  two  hour  old  information.  When  it  gets  to  the 

bunker  it  ends  up  on  the  end  of  a  grease  pencil  for  most  bunkers. 

Now  there  are  a  couple  of  things  I  think  that  we  have 
to  do  in  this  information  business,  fusion  business,  intelligence  support 
business.  The  first  thing  is  we’ve  got  to  recognize  who  is  it  we 
are  supporting.  There  is  a  commander,  there  is  a  decision  maker  for 
every  mission  area  and  he  is  the  one  who  needs  to  give  us  explicit 
guidance  instructions  on  what  he  needs,  what  he  wants.  And  for  each 
mission  on  the  battlefield  the  needs  are  a  little  bit  different.  Secondly, 
the  commander  should  transmit  that  data  to  his  sensor  operators  in 


10 


graphic  form.  In  the  U.S.  Air  Force  today  we  issue  an  air  tasking 
order.  It  generally  is  about  26  pages  of  teletype.  The  Intelligence 
Manager  generally  needs  about  one— half  p3g6  of  data  out  of  that  26 
pages.  By  the  time  you  grind  26  pages  through  the  Communication  Centers 
I  can  guarantee  you  that  the  operations  have  started  before  the  intelligence 
guys  know  about  it.  And  the  only  thing  that  Intelligence  Sensor  Manager 
needs  really  is  a  graphic  picture.  This  is  the  view  of  the  Corps 
Commander’s  mind  of  the  battle  that  is  going  to  unfold.  And  I  get 
a  lot  of  flack  from  my  Air  Force  when  I  say  that  the  Corps  Commander 
is  the  quarterback  for  the  air-land  team,  but  by  God  he  is!  You  are 

not  going  to  conduct  air-to-mud  missions  in  his  area  unless  he  tells 

you  to,  unless  he  asks  you  to.  And  so  his  vision  of  that  battle  needs 
to  be  transmitted  to  that  Air  Commander.  And  they  need  to  discuss 
all  the  areas  where  air  support  is  essential  to  the  Corps  Commander. 

He  needs  intelligence  support.  He  can’t  see  deep  enough.  He  needs 

jamming.  He  doesn’t  have  assets  that  are  high  enough.  He  doesn’t 
have  enough  jammers.  He  needs  deception  support.  He  doesn’t  have 

enough  inventory.  And  obviously  he  needs  close  air  support  and  interdiction 
support.  And  they  need  to  discuss  it  throughout  the  battle  area, 
close-up,  deep  throughout  the  battle  day.  And  this  needs  to  be  translated 

into  action.  But  it  should  start  out  with  a  graphic.  And  when  that 

fighter  wing  has  received  that  graphic  and  has  been  tasked  to  do  an 

interdiction  job,  and  he  finishes  his  penetration  palnning,  he  should 
send  the  Intelligence  Sensor  his  graphic.  This  is  where  I  am  .  going, 

this  is  the  time  I’m  going,  this  is  what  my  database  tells  me  is  there. 

And  you  sensor  operators,  if  you’ve  got  changes,  I  need  to  know  about 

it  whether  they  are  targets  or  threats.  Now  here  is  a  device  that 

I  think  is  going  to  be  very  useful.  It  is  called  Senior  Jade.  Air 
Force  Logistics  Command  is  building  it  now.  They  have  a  prototype 
at  Metro  Tango.  It’s  been  there  for  several  years.  This  is  an  automated 
correlation  system  that  automatically  correlates  different  digital  streams. 
It  will  start  out  correlating  radar  and  COMINT  or  FLINT.  But  in  point 
of  fact,  it  could  be  used  to  correlate  any  type  of  digital  stream 
and  tell  you  very  quickly  from  as  many  different  sources  as  possible 
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what  is  it,  where  is  it,  and  when  did  you  detect  it  and  give  you  some 
identity  information.  The  second  thing  we’ve  got  to  do  is  get  JTIDS. 
It  seems  to  me  that  we  will  never  be  able  to  conduct  combined  arms 
V7arfare  in  a  NATO  environment  until  we  get  an  ability  to  operate  a 
disseminated  command  and  control  system  on  the  battlefield  that  is 
secure  and  jam  resistant.  And  if  this  doesn’t  look  like  the  best 
approach.  I’ll  eat  my  hat  for  sure.  I  think  it  is  the  wave  of  the 
future.  I’m  delighted  to  hear  words  at  the  National  Security  Agency 
that  they  are  planning  on  it  for  their  architecture.  And  finally, 
DARPA  and  NSA  developed  a  little  device  here  that  is  terribly  useful 
to  a  tactical  commander.  It  is  called  Fulcrum  software.  It  essentially 
uses  analog  stored  DMA  maps  and  superimposes  sensor  data  on  top  of 
that.  It  will  change  maps  in  a  third  of  a  second.  The  registration 
of  the  sensor  data  is  more  accurate  than  the  sensors  themselves.  It 
is  off  the  shelf.  We  have  a  little  over  350  systems  out  in  use  today, 
and  it  is  cheap.  I  think  the  TEMPEST  version  is  less  than  $40K  total. 

Now  a  lot  of  people  say  to  me,  well  General  Larson  that  resolution 
is  not  very  good.  Well  my  reply  is,  "Generals  don’t  have  very  good 
eyes."  Furthermore,  "Generals  don’t  need  to  be  looking  at  so  much 
detail."  Let  the  weapons  controllers  take  care  of  that  detail.  This 
will  present  in  quite  good  resolution  on  real  maps  the  symbols  that 
the  general  is  interested  in  keeping  track  of.  So  I  think  those  are 
three  extremely  important  developments  coming  along.  And  with  that 
point  I’ll  quit  and  let  you  hear  something  about  the  Persian  Gulf. 
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BG  William  E.  Harmon,  Army 
Program  Manager,  JTFPO 


I  was  asked  to  discuss  data  fusion  from  a  user  perspective .  My 
military  background  is  principally  that  of  an  intelligence 
officer  in  the  field  of  tactical  intelligence.  As  the  current 
Program  Manager  of  the  Joint  Tactical  Fusion  Program,  I  have 
responsibility  for  a  wide  range  of  Army  and  Air  Force  fusion 
programs,  ^^Ith  the  major  program  consisting  of  the  development  of 

the  Army  A _ ~Source  Analysis  System  (ASAS)  and  the  Air  Force 

Enemy  Situation  Correlation  Element  (ENSCE) .  These  systems  will 
provide  automated  intelligence  support  to  tactical  Army  and  Air 
Force  commanders.  They  will  support  the  operations  and  fire 
support  elements  as  well  as  the  intelligence  off icer  and  will 
provide  an  integrated  picture  of  the  battlefield. 

ASAS/ENSCE  is  an  evolutionary  development  program  because  the 
problem  it  addresses  is  too  big  to  be  solved  in  one  increment. 
ASAS/ENSCE  has  evolved  and  benefits  from  a  number  of  other 
programs  I  also  manage.  Most  of  these  programs  are  already  in 
the  field;  the  resulting  dynamic  user  feedback  makes  my  job  more 
challenging  but  is  essential  if  an  evolutionary  program  is  to 
deliver  useful  systems. 

The  LOCE  (Limited  Operational  Capability  Europe)  and  LENSCE 
(Limited  ENSCE)  systems  are  major  contributors  to  the  ASAS/ENSCE 
program.  These  systems,  derived  from  the  BETA  (Battlefield 
Exploitation  and  Target  Acquisition)  development,  have  evolved 
over  years  of  extensive  field  use.  At  Gallant  Knight  86,  using 
LENSCE,  for  the  first  in  my  military  career  I  saw  three  and  four- 
star  Generals  really  in  sync  on  exactly  what  they  wanted  to  do 
and  how  they  wanted  to  fight  the  battle.  The  LOCE  correlation 
algorithms  are  being  used  in  the  baseline  ASAS/ENSCE. 

I  am  responsible  for  the  Tactical  Simulation  (TACSIM)  System 
which  generates  thousands  of  messages  for  theater— level  exercises 
and  serves  as  the  message  driver  for  ASAS/ENSCE  testing.  The 
Technical  Control  and  Analysis  Center  (TCAC)  systems  provide 
automated  communications  handling,  message  processing  and  data 
retrieval  in  support  of  tactical  SIGINT  operations.  Twenty 
shelters  are  deployed  in  divisions  and  corps  in  Europe  and  Ft 
Bragg,  with  six  more  being  built  for  the  Marine  Corps.  Like 
LOCE/LENSCE,  TCAC' s  success  is  due  in  large  part  to  user 
feedback.  I  have  oversight  of  the  Army's  Microfix  program  which 
uses  Apple  Computers, 

We  are  involved  in  the  fusion  architecture  at  theater  level  to 
insure  that  we  field  systems  that  are  compatible  with  theater 
systems  and  requirements.  Under  the  Fusion  Analysis  Technology 
program,  we  monitor  advanced  technology  efforts  in  your 
laboratories,  NSA,  CIA  and  DARPA,  and  we  evaluate  potential 
applications  for  planned  product  improvements  to  ASAS/ENSCE.  The 
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Digital  Topographic  Support  System  (DTSS) ,  run  by  the  Engineering 
Topographic  Laboratory,  will  provide  line-of-sight  and  terrain 
analysis  map  products  required  by  ASAS/ENSCE.  We  recently 
assumed  responsibility  for  development  of  the  Integrated 
Meteorological  System  (IMETS) . 

The  field  commander  needs  to  know  enemy,  weather,  and  terrain 
The  objective  of  ASAS/ENSCE  is  to  dynamically  integrate  this 
information  and  present  it  to  him  in  concise  terms  of  the  enemy 
situation  and  intentions  so  that  he  can  make  decisions  to 
effectively  employ  combat  power.  Most  importantly,  he  should 
understand  what  he  does  not  know  about  the  battlefield  in  time  to 
intelligently  direct  his  collection  assets. 


Today  we  have  efficient  sensors  but  not  very  efficient  processing 
o  sensor  collections.  Current  largely  manual  systems  get  bogged 
down  in  message  handling.  ASAS/ENSCE  will  provide  quick 
communications  processing  and  dynamic  data  base  storage 
manipulation  and  display.  It  won't  replace  the  analyst,  but  it 
will  provide  analytical  aids  to  facilitate  decision-making. 

An  evolutionary  program  requires  a  sophisticated  audit  of 
requirements  on  a  continuing  basis.  The  physical  ASAS/ENSCE  has 
been  reduced  from _ 10-ton  trailers  to  S-250  shelters  with  smart, 
portable  workstations.  The  baseline  hardware  for  ASAS/ENSCE  has 
been  field  tested.  The  first  three  software  baseline  releases 
have  been  established  from  user  conferences  and  prioritization  of 
functionality  based  on  cost/schedule  risk  and  benefit  analyses. 
The  first  software  release  will  be  field  tested  in  March  of  next 
year.  Requirements  not  incorporated  in  the  baseline  releases 
Will  be  reprioritized,  taking  into  account  emerging  technology, 
or  future  upgrades.  This  is  our  roadmap  to  achieve  the  ASAS 

objective  -  to  allow  the  commander  to  fight  an  effective  air-land 
battle. 
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The  focus  of  this  presentation  will  be  on  what  I  consider  to  be  basic  principles  of  fusion  for  tactical 

commanders.  In  doing  so,  I  will  also  address  two  programs  that  Admiral  Studeman  initiated 
within  the  last  two  years. 

War  is  chaos!  ^e  good  news  i.s  that  it  s  real  hard  for  the  Soviets  to  figure  out  what  we're  doing  in 
the  heat  of  battle;  the  bad  news  is  that  we  can't  often  figure  out  what  we're  doing  either.  My  point 
IS  that  the  opposite  of  fusion  is  confusion-we  can  vastly  improve  tactical  warfare  by  lessening 
factorr"  while  improving  fusion.  To  that  end,  I  have  come  up  with  eleven,  basic  "fusion 

uIt^..ate^cj^nLQfiusm  Unless  warfare  is  completely  automated,  there  will 

always  be  a  decision-making  person  between  the  sensors  and  the  weapon  to  be  delivered  on  the 
target.  Fusion  must  serve  that  person  and  not  just  the  weapons.  No  matter  how  good  the  sensors 
are,  no  matter  how  good  the  communications  are,  no  matter  how  quickly  the  data  comes  to  the 
user,  and  no  matter  how  good  the  processing  algorithms  are-if  the  end  result  is  a  decision  maker 
who  is  confused,  then  the  whole  system  has  failed  and  warfare  is  left  to  instinct  or  chance. 

More  and  faster  always  better.  Keeping  in  mind  the  person  at  the  end 

ot  the  fusion  tram  and  that  this  person  (particularly  in  the  heat  of  battle)  can  only  handle  so  much 
inrormation,  giving  more  data  to  a  person  already  confused  may  not  clarify  the  situation.  Indeed  it 
may  only  worsen  his  decision-mining.  A  key  requirement  that  has  been  brought  up  a  number  of 
tunes  mis  mormng  is  for  clarification— better  data— rather  than  more  or  faster  data.  One  of  the  wavs 
to  clarify  is  to  tailor  the  tactical  picture. 

Xactical  fusion  must  be  tailored  The  right  user  must  get  the  right  information  in  the  right  amount 
at  the  nght  time;  anyAmg  else  iiicreases  confusion.  The  system  providing  that  information  must  be 
flexible  in  order  to  adjust  the  tailoring  to  the  situation  at  hand,  rf  possible,  that  tailoring  should  be 
done  by  the  decision  maker;  if  not  possible  in  the  heat  of  battle,  it  must  be  done  by  some 
operational  intelligence  organization  "fronting"  for  the  tactical  commander.  The  Intelligence 

began  for  this  very  reason.  ISS/AF,  a  joint 
O  NAV  and  CMC  effort  led  by  Admiral  Studeman,  is  an  effort  to  improve  our  ability  to  provide 
support  at  existing  shore- based  and  afloat  sites  and  provide  new  tools  for  those  going  ashore.  The 
basic  ISVAF  problem  is  that  the  tactical  commander  knows  his  tactical  information  needs  best  but 
has  the  least  capability  to  coordinate  tactical,  theater  and  national  sensors.  He  has  the  least 
capabdity  to  process  the  best  amount  of  data  avadable  to  him,  and  even  if  he  could,  he  has  the  least 
capability  to  transform  the  information  into  a  product  his  weaponry  needs.  ISS/AF  seeks  to 
provide  those  capabilities  in  a  tiered  fashion  to  tactical  forces  ashore. 

environmental  d.tp 

ISS/AF  has  highlighted  this  requirernent.  All  forces,  even  those  in  space,  fight  relative  to  the 
earth.  Accordingly,  tactical  informational  must  be  presented  to  tactical  decision  makers  in  the 

SSnf Obviously,  airplanes  can't  fly  through  mountains  and  tanks 
don  t  normally  cross  lakes.  These  factors,  combined  with  environmental  data  and  the  efforts  of  the 
environment  must  be  presented  to  tactical  commanders.  For  example,  has  the  weather  been  cold 
enough  long  enough  for  the  lake  to  have  frozen  sufficiently  for  tanks  and  trucks  to  drive  across? 

la^icalji^on_Bmstjn4!MeJIMshed^  Tactical  decisions  must  include  reference  data 

such  as  order  of  battle,  installation  data  and  weapons  characteristics.  It  is  not  enough  to  say  to  a 
taaical  commander  you  have  two  Boghammers  closing  rapidly  at  030."  To  make  a  balanced 
cohesive  tactical  decision,  he  needs  to  hear  something  like  "there  are  two  out  of  their  four 

five-inch  rockets,  closing  rapidly  at  030."  The  commander  needs 
to  hdve  the  situation  presented  to  him.  in  the  context  of  the  whole  threat. 
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Tactical  fusion  must  allnw  fnr  multi-level  secure  data  dissemination.  The  point  here  is  simply  that 
ther^are  many  sensitive  sources  with  data  that  could  significanfly  lessen  confusion  in  the  decision 
maker’s  thought-data  not  normally  shared  with  some  levels  of  command  dueinto  the  system  and 
^vn  to  Slower  access  levels  Uen  it’s  needed  and  in  a  timely  manner?  How  do  we  give  him 
confidence  in  the  data  without  revealing  sensitive  sources? 

fndnn  mnsi  ho  coordinated  end-to-end,  from  the  sensor  to  the  weap^ 
that  fusion  is  not  limited  to  a  processing  system  or  a  series  of  systems;  it  occurs  all  along  the  way 
from  a  smart  sensor  to  a  decision  maker  to  a  smart  weapon  and  it  must  be  coordinated  accordingly. 
We  can  have  tons  of  high  quality  sensor  data  wonderfully  fused  and  sent  to  the  decisicm 
but  if  it  all  goes  through  a  75  baud  circuit  which  is  backlogged  six  hours,  the  system  has  failed. 
TOfoIrdiSnZf;  Lclude  smart  reporting,  protocols,  — “^ZSoS'mrt 
interfaces,  data  base  structures,  and  man-machine  interfaces.  A  basis  for 

include  smart  reporting,  protocols,  communications  circuits,  system  interfaces,  data  base 
s^cmresTd  man-machine  interfaces.  A  basis  for  this  coordination  is  Information  Management 
orlNFOMAT  initiated  by  Admiral  Studeman  to  improve  end-to-end  coordination  “ 
with  sensor  managers,  command  and  control  entities,  intelligence  centers,  and  . 

cannot  afford  to  fight  in  the  dark  or  pay  for  sophisticated  sensors  that  produce  unused  data. 

Tf  ^»nfncinn  iQ  1p.«;sp,neH  bv  commonalitv.  watch  out  for  exploitable  loopholes.  This  fusion  factor 

is  really  a  warning.  We  are  hearing  more  and  more  about  comnion  hardware,  common  software, 

coSmSL  afoS^.  common  data  bases,  etc.,  usually  promot^  for  ^aranteed  mteroperabihty  or 

lowered  life  cycle  maintenance  costs.  It’s  important  to  keep  in  mind  that 

two-edged  swid;  if  we  have  complete  commonality  and  there  s  at  least  one 

that  is  successfully  deceived,  then  we  will  all  be  deceived  and  all  go  over  the  cliff  together. 

n^r>ictr>n  maVers  must  not  only  understand  the  situation,  they  must  believcjL 
overstated.  Remember  the  person  in  the  link  from  the  sensors  to  die  weapons;  no  matter  how  go^ 
the  data,  no  matter  how  well  it’s  presented  to  be  understood-if  it  s  not  ' 

We  must  not  try  to  dazzle  or  impress  the  person  at  the  other  end,  we  must  educate  him.  This  is 
particularly  true  today  when  we  have  so  much  data  automatically  proce^ssed  and  displayed,  e 
mental  "distance”  between  the  person  and  the  raw  data  is  approaching  mfinity  as  we  are  forc^  to 
trust  automated  processing  of  information  we  never  see  based  on  software  we  don  t  understand^ 
There  must  be  a  way,  no  matter  how  complicated  the  fusion  system,  for  the  user  to  say  to  the 
system  "prove  it;  show  me  how  you  came  up  with  that  conclusion. 

K'nnwinp  intentions  reduces  confusion.  Obviously,  a  decision  maker  s  job  is  much  easier  if  he 

Lows  the  future-the  enemy's  intentions.  We  were  very  successful  in  “^““8 

warfare  in  the  Atlantic  in  World  War  II  not  because  we  knew  where  the  German  U-boats  wm 

every  minute  of  the  day  but  by  reading  their  "maU"  and  knowing  where  they  were  going  to  Md 

S  SS  were  going  m  be  there.  And  of  course,  even  if  we  could  have  ^cked  them  constantly. 

coirectly  Lowing  intentions  is  better  than  extrapolating  histone  tracking  ^ta  for 

on  a  target.  The  goal  is  to  be  able  to  read  a  target-to  predict  through  fusion  and 

through  "reading  his  mail."  However,  this  too  is  a  two-edged  sword,  as  peyle  do  their 

tactics  and  intentions.  Intentions  must  be  fused  with  current  locational  data  and  capabihties. 

Dprision  makers  don’t  live  in  the  past;  their  decisions  are  about  the  future^  'The  last  "fusion 
factor "  It  is  important  to  remember,  as  Admiral  Studeman  pointed  out,  that  intellipnce  office 
^?note  Sy^historians.  Their  purpose  is  not  solely  to  catalog  the  past  down  to  the  second  and 
tell  us  what  happened.  Their  job  is  to  predict  the  future  so  that  the  decision  maker  can  raalre  a 
decision  about  the  future-not  about  the  past.  The  fusion  system  must  not  lose  sight  of  that.  The 
tactical  commander  must  initiate  defensive  measures  before  a  threat  is 

his  target  offensively.  The  fusion  process  form  end  to  end  must  allow  and  even  promote  future 
extrapolation. 

That’s  all  But  for  purposes  of  this  symposium,  it’s  important  to  reinember  these  basic  fu^sion 

principles  while  diseasing  complex  and  state-of-the-^t  ways  to  fuse  data  and 

processing.  They  were  identified  from  the  vantage  point  of  the  tactical  user-the  tactical  decision 

maker  for  whom  we  build  fusion  systems. 
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TASK  FORCE  DEFINITION  OF  IDEAL  COMMAND 

AND  CONTROL 
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and  control  systems  is,  in  the  end,  to  provide  assurance 
that  orders  are  received  as  originally  intended  with  follow¬ 
up  in  a  timely  fashion,  which  can  make  the  difference 
between  winning  and  iosing  wars." 


MAJOR  AREAS  OF  COMMAND  AND  CONTROL 

SYSTEM  SHORTFALLS 
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There  is  too  much  emphasis  on  the  deveiopment  of 
completeiy  new  systems  over  the  evolution  of  the  very 
comprehensive  already  in-place  systems. 


MAJOR  AREAS  OF  COMMAND  AND  CONTROL 
SYSTEM  SHORTFALLS  (Continued) 
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evaluation  exercises. 


MAJOR  AREAS  OF  COMMAND  AND  CONTROL 
SYSTEM  SHORTFALLS  (Continued) 
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MAJOR  AREAS  OF  COMMAND  AND  CONTROL 
SYSTEM  SHORTFALLS  (Continued) 
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We  must  be  willing  to  develop  the  research  specialists 
necessary  to  ensure  that  we  are  at  the  leading  edge  of 
command  and  control  technology. 


MAJOR  AREAS  OF  COMMAND  AND  CONTROL 
SYSTEM  SHORTFALLS  (Continued) 
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RECOMMENDATION  AREAS 
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RECOMMENDATION  AREAS 
(Continued) 
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RECOMMENDATION  AREAS 
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CINCs. 


RECOMMENDATION  AREAS 
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Embedded  training  and  programs  shouid  be  developed  to  enable 
training  to  proceed  in  the  most  realistic  environments  as  well  as 
to  provide  improved  feedback  to  developers. 


UNDER  SECRETARY  OF  DEFENSE  RESPONSE 
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JOINT  DIRECTORS  OF  LABORATORIES  (JDL)  required  to 
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DATA  FUSION  RESEARCH  MANAGEMENT 


DR.  DANIEL  F.  WIENER  II 


UNIVERSITY  OF  VIRGINIA 
SCHOOL  OF  ENGINEERING  AND  APPLIED  SCIENCE 


INTRODUCTION 

The  fusion  of  data  to  provide  informa¬ 
tion  is  not  a  new  endeavor.  It  has  been 
present  throughout  recorded  time.  Hereto¬ 
fore,  it  was  always  accomplished  by  a 
living,  thinking  organism. 

War  fighters,  in  particular,  have  long 
recognized  the  need  for  information,  not 
data,  but  have  had  a  difficult  time  in 
describing  the  totality  of  their  require¬ 
ment.  With  the  proliferation  of  elec¬ 
tronics-based  sensors,  the  streams  of  data 
have  gone  from  light  mists  to  raging 
torrents  eclipsing  the  capability  of  manual 
approaches.  As  such,  the  information 
management  community  has  started  down 
several  grand  and  glorious  paths  to  use 
electronics  to  automatically  solve  the 
•'problem"  automatically,  i.e.,  deriving 
information  from  data.  While  one  might  be 
expected  to  state  that  the  problem,  as  well 
as  the  solution,  is  routed  in  technology, 
it  is  this  paper's  premise  that  the  end  can 
only  be  accomplished  through  a  judicious 
use  based  on  a  realistic  research  approach 
with  sound  research  management. 


BACKGROUND 

Data  fusion  research  management  has 
been  thought  of  as  an  oxymoron.  Data 
fusion  research  and  data  fusion  management 
appear  to  be  conflicting  concepts.  It  was 
not  conceived  to  be  that  way  but  evolved 
because  of  what  is  commonly  called  the 
"Quest-For-Research-Gold. " 

The  Quest-For-Research-Gold  concept 
can  be  described  as,  "if  there  is  funding 
for  a  research  area,  I  want  to  get  my  fair 
share."  Fair  share  is  defined  by,  and  is 
in  the  eyes'  of,  the  beholder.  Like 
Artificial  Intelligence  a  few  years  past 
and  Artificial  Neural  Systems  today,  data 
fusion  is  a  "hot"  topical  research  area. 
Therefore,  everyone  within  the  government 
research  community  wants,  and,  I  might  add, 
can  justify,  a  piece  of  this  research 
dollar  pie.  Furthermore,  no  consultant, 
industrial  organization,  or  academic 
institution  wants  to  be  left  out.  The 
result  is  research  chaos.  However,  as 


Nietzsche  said,  "Out  of  chaos  comes 
order."  With  the  emergence  of  Joint 
Directors  of  Laboratories  -  Data  Fusion 
Subpanel  (JDL-DFS)  and  the  Joint  Tactical 
Fusion  Program  (JTFP) ,  as  well  as  inter¬ 
national  organizations  like  the  SHAPE 
Technical  Center  Research  Panel ,  order  is 
in  sight. 


DEFINITION 

The  history  of  the  data  fusion 
research  is  fraught  with  many  long-standing 
arguments  based  on  semantics.  JTFP  and 
JDL-DFS  have  sought  to  resolve  this  issue 
with  the  drafting  of  candidate  definitions. 
For  the  purpose  of  this  discussion,  the 
following  JTFP  definitions  are  used: 

Analysis:  A  series  of 
technical/cognitive  actions  taken  to 
convert  collected  data  into  reports. 

Correlation:  The  process  that 

combines  and  associates  reports/ob¬ 
servables  from  the  same  or  different 
sources  that  refer  to  the  same 
battlefield  entity,  to  form  a  single 
record  that  consists  of  the  best 
available  information  on  that  entity. 


Aggregation:  The  process  of 

combining/associating  reports/ ob¬ 
servables,  to  identify  parent 
organizations/subordination  based  on 
inferences  drawn  from  the  presence  of 
its  component  parts. 

Fusion:  The  function  of  providing  an 

Intelligence  Product  from  merged 
reports  based  on  multiple  sources  and 
systems  from  multiple  Intelligence 
disciplines  that  supports  the  situa¬ 
tion  analysis  and  user  comprehension 
of  battlefield  dynamics. 

While  these  definitions  may  not  be 
perfect,  they  were  derived  through  an 
Adelphi  process  with  the  military  and 
technical  leadership  in  1983  under  Joint 
Tactical  Fusion  Program  sponsorship. 
Although  longer  than  necessary,  they  have 
served  the  test  of  time  during  a  period  of 
increased  awareness  in  data  fusion  tech- 
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nologies  and  systems.  No  such  definition 
exists  for  all  source  data  fusion.  One 
man's  all  source  data  fusion  is  another 
man's  multi-source.  The  issue  of  def¬ 
initions  is  further  complicated  when  one 
considers  the  use  of  preprocessors  —  not 
withstanding  ownership  of  data  as  issues. 
However,  the  important  issue  is  to 
establish  a  basis  of  context  for  discus¬ 
sions  . 


PREJUDICES 

The  prejudices  underlying  this 
article  are  based  on  almost  two  decades  of 
work  in  tactical  warfare  technology, 
particularly  concentrating  on 
interdiction/strike  weaponry  and  most 
recently  as  the  Deputy  Program  Manager  and 
Technical  Director  of  the  Joint  Tactical 
Fusion  Program. 

Initial  involvement  with  what  was 
described  as  fusion  was  with  the  U.S.  Navy 
during  Vietnam.  VQ  squadron  support  was 
SIGINT-based  with  acoustic  augmentation  to 
provide  a  situation  assessment  of  the 
enemy,  as  well  as  real-time  missile  and 
enemy  aircraft  warnings  to  our  strike 
aircraft.  The  shrouds  of  secrecy  prevented 
widespread  dissemination  at  the  time. 

Fusion  was  defined  by  RADM  Roy  M.  Isaman 
(deceased)  as  "what  you  do  to  get  useful 
information  from  data  —  but  in  many 
different  forms."  The  same  data  would  be 
"fused"  in  many  different  ways  with  many 
different  sources  to  provide  as  many 
different  products. 

Subsequent  activity  also  involved  the 
use  of  heretofore  "green  door"  products  to 
further  tactical  warfare  and  tactical 
weaponry.  One  of  these  was  the  Cruise 
missile,  another,  more  grandiose  concept 
was  Assault  Breaker.  The  original  Assault 
Breaker  concept  was  based  on  three  ele¬ 
ments.  A  large,  stand-off  airborne  radar; 
a  modular  warhead,  "long-range"  missile; 
and  a  glue  to  interface  the  two.  This  glue 
was  reminiscent  of  the  discussions  on 
"cosmic  glue"  used  to  hold  the  elements  of 
an  atom  together.  As  pi-mesons,  mu -mesons 
and  gluons  were  discovered,  real-time  data 
fusion  systems  were  also  discovered. 


BACKGROUND 

To  understand  the  present,  we  must 
look  to  the  past,  and  the  past  is  the 
Battlefield  Exploration  and  Target 
Acquisition  Program,  BETA. 

The  tri-service  BETA  Program  was 
established  by  the  Office  of  the  Secretary 
of  Defense  in  1977  as  a  technical  demon¬ 
stration  project  with  an  objective  to  par¬ 
ticipate  in  the  REFORGER-80  exercise.  The 
purpose  was  to  support  battlefield  com¬ 
manders  in  the  execution  of  the  air/land 
battle,  particularly  with  respect  to 
resource  allocation,  maneuver  of  forces, 
and  targeting.  The  approach  of  BETA  was  to 
correlate/fuse  multi-disciplinary  sensor 
products  (alphanumeric  messages)  into  a 
coherent  graphic  picture  of  the  battlefield 
with  an  implementation  based  on  commercial 
hardware  and  the  development  of  correla¬ 
tion  software  to  accomplish  first-order 
data  fusion  that  could  be  used  to  support 


the  development  of  individual  service  data 
fusion  systems.  The  BETA  project  ran  into 
continual  problems  with  interpretation  of 
requirements.  As  such,  BETA  was  unable  to 
participate  in  REFORGER.  BETA  was  of¬ 
ficially  concluded  in  February  1981  when 
the  the  two  test  beds  were  accepted  at  the 
contractor's  plant  and  subsequently 
deployed  to  the  Army  at  Ft.  Hood,  Texas, 
and  to  the  Air  Force  at  Hurlburt  Field, 
Florida.  These  test  beds  were  to  be  used 
for  user  testing  at  these  sites  to  refine 
requirements  and  support  the  subsequent  All 
Source  Analysis  System/Enemy  Situation 
Correlation  Element  (ASAS/ENSCE)  develop¬ 
ment. 

A  composite  capability  of  BETA 
fashioned  from  the  contractor's  software 
development  facility  and  some  of  the  Ft. 
Hood  and  Hurlburt  assets  was  deployed  to 
Europe  in  October  1982.  This  BETA  Derived 
System  for  the  U.S.  European  Command 
system,  known  as  Limited  Operational 
Capability  Europe  (LOCE) ,  was  to  support 
both  U.S.  and  NATO  forces.  In  November 
1987,  the  LOCE  role  was  expanded  to  serve 
as  a  gateway  to  NATO.  The  Hurlburt  test 
bed  was  repackaged  and  deployed  to  Shaw  AFB 
in  1985  to  support  the  Gallant  Knight 
exercise  in  February  1986.  The  exercise 
was  extremely  successful,  resulting  in  the 
Air  Force  Commander,  Lt.  General  Kirks, 
commenting  that  "it  was  the  best  intel¬ 
ligence  support  he  had  ever  received."  As 
a  result,  the  system  is  in  the  process  of 
being  upgraded  to  serve  as  an  interim 
ENSCE  capability.  A  like  system  has  been 
replicated  for  the  12th  TIS  at  Bergstrom 
Air  Base. 

JTFP  was  established  originally  in 
1981  as  an  outgrowth  of  the  old  BETA 
project.  Many  jurisdictional  battles 
ensued  with  government  agencies, 
contractors  and  congressional  staffs  all 
having  the  "correct  solution."  As  a 
result  of  a  recommendation  of  the  study 
led  by  Brigadier  General  Phillip  Mason 
USA,  a  new  JTFP  was  established  as  a 
Congressionally  directed  acquisition  in 
1983  to  bring  together  all  military 
tactical  intelligence  data  fusion  develop¬ 
ment  activities.  The  program  has  a  unique 
reporting  structure.  It  reports  currently 
to  a  Joint  Oversight  Group.  This  group  is 
comprised  of  the  same  members  one  would  see 
at  a  Service/Defense  Research  Board  with 
representation  from  the  Assistant  Secretary 
of  Defence  for  C3I. 

Ten  major  projects  under  the  JTFP  are 
taking  place.  The  keystone  project  is  the 
All  Source  Analysis  System/Enemy  Situation 
Correlation  Element  (ASAS//ENSCE) .  JPL  is 
the  major  contractor  to  this  joint  Army-Air 
Force  program  with  two  names.  The  remain¬ 
ing  projects  include  the  Quick  Reaction 
Multicolor  Printer,  an  effort  to  put 
essentially  a  Xerox  machine  on  the  battle¬ 
field;  the  Intelligence  Digital  Message 
Terminal,  AN/PSC-2 ;  the  Litton  handheld 
device  used  for  plugging  digital  data  into 
the  overall  system;  the  Army's  Technical 
Control  and  Analysis  Centers,  the  first 
automation  support  to  the  Army's  CEWI 
functions  at  Division  and  Corps,  as  well 
as,  the  First  and  Second  Radio  Battalion 
for  the  U.S.  Marine  Corps;  the  Digital 
Topographic  Support  System  (DTSS) ,  managed 
by  the  Engineering  Topographic  Laboratory 
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to  provide  digital  maps  to  the  battlefield; 
the  Integrated  Meteorological  System 
(IMETS) ,  managed  by  the  Atmospheric 
Sciences  Laboratory  to  provide  an  in¬ 
tegrated  weather  product;  the  previously 
discussed  BETA  Derived  Systems,  Limited 
Operational  Capability-Europe,  LOCE, 
located  at  Ramstein  Air  Force  Base  under, 
the  operational  control  of  US  EUCOM,  repre¬ 
sents  an  interface  into  the  Allied  Theatre 
of  Operations  for  multisensory  data  fusion. 


Also  under  derived  systems  are  the 
LENSCE,  Limited  Enemy  Situation 
Correlation  Element  Systems.  These 
systems  provide  near  term  automation 
support  to  the  intelligence  functions  at 
the  TACC  for  the  9th  Tactical  Intelligence 
Squadron  and  the  12th  Tactical  Intelligence 
Squadron;  the  Intelligence  Fusion  Analysis 
Technology  Validation  Project,  the  6.3b 
advanced  technology  bridge  and  TACSIM,  the 
tactical  simulator.  The  TACSIM  program  was 
placed  under  the  JTFP  umbrella  because  of 
its  use  as  a  driver  for  testing  and 
training  activities  for  the  ASAS/ENSCE 
program.  Its  primary  purpose,  however,  is 
to  provide  simulation  support  for  intel¬ 
ligence  operations  at  U&S  command  exer¬ 
cises.  To  preclude  duplication,  and 
insure  connectivity.  Congress  directed 
that  all  data  fusion  activities  must  be 
accomplished  under  the  JTFP  umbrella.  The 
Army  fully  embraced  this  concept  by 
placing  under  a  single  umbrella  all  of  the 
projects  necessary  to  assure  an  integrated 
weather  enemy  and  terrain  product  to  the 
commander. 

The  Joint  Tactical  Fusion  Program 
neither  had,  nor  desired  to  have,  a 
research  laboratory.  As  such,  JTFP  worked 
with  the  services  to  sponsor  the  Joint 
Directors  of  Laboratories,  Data  Fusion 
Subpanel  to  bring  together  overlapping 
research  areas.  This  JDL  initiative  also 
takes  its  lineage  from  the  Herman  Report  of 
1982  which  identified  52  data  fusion 
centers  within  DOD  where  data  fusion  or 
related  research  is  conducted.  It  is, 
therefore,  obvious  that  the  Data  Fusion 
Subpanel  program  can  act  as  a  central 
clearing  house,  or  point  of  focus,  for 
coordinating  individual  services  and 
agencies  data  fusion  research  and  develop¬ 
ment  efforts.  The  Data  Fusion  Subpanel 
directorate  is  now  asking  questions 
regarding  its  relationship  to  other  parts 
of  the  Department  of  Defense  establishment, 
and  how  it  can  gain  the  necessary  leverage 
and  visibility  for  its  programs.  This  may 
be  an  important  question  for  bureaucratic 
purposes,  but  is  not  necessary  to  the  well 
being  of  their  mission.  The  better 
question  is  "how  to  present  a  unified 
coordinated  research  program.” 


RESEARCH  GOAL 

What  is  the  goal  of  data  fusion 
research?  This  can  best  be  addressed  by 
looking  at  the  Army  All  Source  Analysis 
System/ Air  Force  Enemy  Situation 
Correlation  Element  (ASAS/ENSCE) . 
ASAS/ENSCE  is  a  multi-billion  dollar 
program  to  automate  data  fusion.  The 
purpose  of  the  program  is  to  provide 
timely,  accurate,  organized  information 
that : 


Facilitates  decision  processes; 

Accrues  maximum  benefit  from 
assets ; 

Clarifies  and  adds  depth  to  the 
commander's  view  of  the  battle¬ 
field; 

Gives  a  common  view  of  the 
air/land  battle  to  the  ground 
and  air  commanders;  and 

Generates  an  opportunity  to 
seize  and  retain  the  initiative. 


Thus,  the  commander  is  placed  in  the 
"driver's  seat,”  That  is,  to  provide 
the  right  information  at  the  right 
time  and  in  the  right  format.  The 
goal  is  to  give  a  common  picture  of 
the  battlefield  and  really  bring 
intelligence  to  a  service  function  to 
the  battle  commander.  However,  this 
is  a  monumental  effort.  But  like  all 
programs,  one  must  start  someplace; 
one  must  have  intermediate  milestones; 
one  must  finish.  When  automating  a 
function  that  heretofore  has  been 
manual,  it  is  necessary  to  take  it  in 
bite-size  steps.  One  cannot  eat  the 
whole  elephant  at  once.  One  must  do 
him  one  bit  at  a  time,  and  this  one  is 
an  elephant. 

As  a  result,  ASAS/ENSCE  developed  a 
policy  and  built  a  program  around  the 
concept  of  evolutionary  development,  where 
evolutionary  development  was  defined  as: 

The  process  of  incremental 
development  and  fielding  of  a 
system  with  maximum  user  par¬ 
ticipation.  The  process  beings 
with  the  development  and  fielding 
of  a  minimum  capability  baseline 
system  with  a  lead  field  user. 

Feedback  based  on  system  perfor¬ 
mance  in  an  operational  environ¬ 
ment  guides  the  incremental 
growth  from  the  baseline  to  the 
full  objective  system  capability. 


The  philosophy/ approach  is  summarized  as 
build  a  little,  test  a  little,  analyze, 
build  a  little  more,  test  a  little  more, 
analyze,  work  it  through  in  very  close 
contact  with  the  user.  If  you  were  to 
think  of  our  community  as  a  commercial ^ 
organization,  it  is  staying  in  touch  with 
our  customer  and  giving  him  what  he  needs. 


There  are  major  benefits  to  this 
approach:  the  ability  to  incorporate 

advanced  technology  solutions  as  they 
become  available,  such  as  lighter,  smaller, 
more  powerful  computers  and  AI.  AI,  in 
particular,  allows  for  problem  definitions, 
problem  solving,  and  problem  understanding 
in  this  incremental  process  and  for  incor¬ 
poration  of  that  knowledge  into  the^ 
ASAS/ENSCE.  A  potential  side  benefit  of 
this  is  that  the  shock  on  the  system,  that 
is,  U.S.  Army  and  Air  Force  operations 
worldwide,  will  be  minimized  by  allowing 
their  concepts  of  operation  to  evolve  with 
the  development  of  the  system. 
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This  philosophy  is  in  conjunction 
with,  the  entire  multi-million  dollar 
DARPA  Machine  Intelligence  research  area, 
whose  goal  is  to  develop  fundamentally  new 
capabilities  for  information  processing 
that  make  computers  intelligent  through  a 
process  of  demonstrating  computational 
equivalents  to  human  cognition,  perception, 
and  action.  In  the  data  fusion  research 
area,  the  following  three  major  challenges 
were  defined  as: 

-  Automation  of  the  tactical 
battlefield; 

Fusion  of  multi-source  data;  and 

Dissemination  of  product. 

The  implied  primary  needs  are:  the 
integration  with  deterministic  systems  and 
the  cooperation  with  other  intelligent 
systems.  The  secondary  needs  are: 
learning,  uncertainty  management,  and 
causal  reasoning.  As  such,  the  magnitude 
of  the  dollars  has  resulted  in  a  virtual 
stampede  to  have  one's  own  data  fusion 
research  program  and  system. 


CONGRESSIONAL  INVOLVEMENT 

Congress  had  a  heavy  hand  in  the 
establishment  of  the  BETA  program  and  its 
successor,  Joint  Tactical  Fusion.  HPSCI, 
HAC,  and  SASC  were  the  prime  players. 
Studies  by  the  Congressional  Surveys  and 
Investigation  staff  in  the  late  1970 's 
found  that  the  fusion  of  intelligence  data 
into  a  format  having  utility  for  the 
combat  commander  was  one  of  the  weakest 
links  in  our  tactical  intelligence  system. 
Congress  cut  the  1978  budget  for  fusion 
centers  by  10  percent  to  encourage  program 
consolidation.  A  subsequent, 
Congressionally  directed  study  done  by  the 
Institute  for  Defense  Analysis  showed  52 
fusion  center  projects  in  progress.  BETA, 
its  successor,  Joint  Tactical  Fusion,  and 
the  Joint  Directorate  of  Laboratories 
Subpanel  on  Data  Fusion  were  natural 
outgrowths  of  the  Congressional  interest 
and  concerns.  When  the  BETA  software 
proved  insufficiently  stable  to  support  the 
European  Demo  in  the  fall  of  1980,  Congress 
directed  that  BETA  be  concluded  and  a  Joint 
Tactical  Fusion  program  be  organized  with 
the  initial  manpower  drawn  from  the  BETA 
program.  Joint  Tactical  Fusion  is  building 
ASAS/ENSCE  for  earliest  possible  delivery 
and  to  capitalize  on  BETA  and  related 
fields  and  ongoing  developments. 

This  implies  that  the  solution  to 
Data  Fusion  Research  management  may  not 
exist,  but  there  are  questions  and  lessons 
that,  if  answered  and  assimilated,  can 
provide  a  road  map.  But  we  must  finish 
with  the  definition  of  the  problem. 

Consider  General  Hagaman's  question 
of  last  year's  symposium,  "Is  Data  Fusion  a 
discipline  or  a  community?"  General 
Hagaman  pointed  out  that  Webster  defines 
community  as  it  relates  to  professionals  as 
"a  body  of  persons  of  common  and  special 
professional  interests  scattered  through  a 
larger  society."  The  data  fusion  par¬ 
ticipants  are  members  of  a  viable,  expand¬ 
ing,  maturing  community,  but  to  what  end? 
The  institutionalization  of  a  data  fusion 


community  and/or  formalization  of  the 
structure  by  forming  a  data  fusion  profes¬ 
sional  association  does  not  aid  or  abet  the 
development  and  fielding  of  a  data  fusion 
capability.  This  symposium,  as  Co¬ 
ordination  Task  4  of  the  Data  Fusion 
Subpanel  Program,  has  resulted  in  an 
increased  awareness  of  what  we  can  and 
cannot  do  while  serving  as  a  sounding 
board  to  assure  cooperation  without 
duplication.  As  such,  the  creation  of  a 
new  body  is  not  required. 

But  what  is  the  problem?  The  problem 
is:  how  do  we  manage  data  fusion  research 

with  the  goal  of  achieving  near-term 
fielding  of  systems.  The  hov;  is  a  complex 
issue  with  no  single  correct  solution.  The 
lessons  learned  from  the  past  suggest  that 
central  management  points  are  required. 
Organizations,  such  as  the  Joint  Tactical 
Fusion  Program  and  Joint  Directors  of 
Laboratories  Data  Fusion  Subpanel,  can 
serve  such  purposes. 


SUMMARY 

In  summary,  the  fusion  of  data  into 
information  is  a  cognitive  process  older 
than  man.  We  like  to  say  that  man  is  the 
"thinking  animal,"  but  as  we  learn  more 
about  the  thought  process  we  realize  that 
the  combining  of  sensory  data  into  informa¬ 
tion  for  the  subsequent  control  of  actions 
can  be  a  multi-leveled  activity.  We  are 
currently  at  the  level  of  the  slug,  which 
has  caught  the  fancy  of  many  researchers  in 
Artificial  Neural  Systems.  The  slug  cannot 
be  placed  on  the  same  level  as  a  dog,  let 
alone  man.  However,  it  does  have  sensors 
which  input  data  into  a  multi-source  fusibn 
process  to  control  its  actions  and 
reactions.  The  lesson  we  must  learn  is 
this  is  not  a  simple  problem  that  will  be 
solved  overnight.  All  source,  multilevel 
data  fusion  is  a  goal  —  achievable  but  not 
in  the  near  term.  The  triumvirate  of 
government,  industry,  and  academia  must 
not  over-require,  over-specify ,  over¬ 
commit,  or  over-sell.  That  is  not  to  say 
that  we  cannot  push  the  state-of-the-art 
aggressively.  However,  only  through 
honesty  and  joint  efforts  will  the  des¬ 
perately  overarching  goal  be  achieved. 
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ABSTRACT 


This  paper  describes  the  requirements  of  the  recently  approved  NCCS (A)  Program  based  on  an 
appreciation  of  its  future  operational  environment.  It  stresses  Decision  Support  applications 
to  the  data  correlation  requirements  of  the  system  as  it  is  currently  planned  for  installation 
on  CV/CVN  and  LCC  class  ships. 

The  NCCS(A)  consolidates  all  of  the  processes  required  to  support  the  historically  segregated 
and  largely  manual  functions  of  warfare  assessment,  planning,  and  control  into  a  single 
architecture.  Advanced  techniques  for  data  correlation,  fusion  and  storage;  and  of  Information 
management,  analysis,  retrieval,  sharing,  and  display  will  be  incorporated  into  the  system  to 
facilitate  decision  making  by  the  Battle  Force/Group  (BF/BG)  Commander  and  his  staff. 

The  paper  describes  the  incremental  integration  of  standardized  components  (such  as  the  advanced 
color  workstation)  into  a  group  decision  support  complex  within  the  Tactical  Flag  Command  Center 
(TFCC).  It  presents  the  integrated  data  fusion  and  spectrum  management  components  of  the 
NCCS(A)  system.  A  top  level  discussion  of  the  system  architecture  is  followed  by  a  detailed 
description  of  each  of  its  information  management  components.  Particular  attention  is  paid  to 
the  dual  security  bus  that  connects  its  distributed  components  throughout  the  command  and 
control  spaces  of  the  platform. 
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INTRODUCTION 

Given  the  current  state  of  systems 
development  practices,  the  contemporary 
Program  Manager  has  three  broad  management 
initiatives  which  he  must  achieve  if  the 
management  of  his  Program  is  to  be  suc¬ 
cessful.  Creating  a  viable  management 
platform  requires:  (1)  the  standardization 
of  devices,  (2)  the  reconciliation  of  con¬ 
flicting  development  policies,  and  (3)  the 
revitalization  of  the  programmatics  of 
systems  development. 

The  Program  Office  has  approached  this 
requirement  by  combining,  in  a  single 
Program  Office,  the  four  projects:  TFCC, 

ACS,  EWCM,  and  NIPS  and  by  creating  a  unique 
development  strategy  for  the  amalgamation 
of  the  most  valuable  operational  elements 
of  each.  From  this  nucleus  of  related 
projects  PMW162  has  developed  a  single  co¬ 
hesive  application  of  Information  synthesis 
and  management  tools  called  the  Naval 
Command  and  Control  System  -  Afloat  NCCS(A). 
This  paper  presents  the  Operational  Require¬ 
ments,  Architecture,  Subsystem  Elements, 
Development  Strategy,  Operational  Scenario 
and  a  discussion  of  some  of  the  newer 
technological  innovations  of  that  system. 

OPERATIONAL  REQUIREMENTS 

The  operational  requirements  of  NCCS(A) 
are  for  mechanisms  that  allow  control  of 
naval  forces  in  not  only  a  physically 
diverse  environment  but  also  in  one  wherein 
the  elements  of  control  are  spread  across 
three  distinctly  different  media:  physical, 
cognitive,  and  spectral.  The  physical 
medium  is  the  "hardkill",  or  destructive, 
component  of  the  combat  and  is  the  one 
medium  that  has  been  most  frequently 


addressed.  The  cognitive  medium  is  the 
information,  or  entropic,  component  of  the 
combat  and  has  only  partially  been  addressed 
by  land  based  systems.  The  spectral  medium 
is  the  "soft-kill”,  or  plasmatic,  component 
of  the  combat  and  has  never  been  addressed 
as  an  automated  system.  Thus,  NCCS(A) 
exists  because  Asset  Management  for  the 
Battle  Force  has  become  intellectually 
diversified  in  that  it  now  extends  to  the 
control  of  electromagnetic  and  information 
resources  as  well  as  of  the  combatant 
weapons . 

Battle  management  also  encompasses 
control  of  two  different  types  of  activities 
each  of  which  requires  a  different  set  of 
behavioral  responses.  There  are  those  that 
occur  in  the  First  Domain,  the  actual  combat 
itself,  and  which  could  be  classed  as  "para- 
spontaneous"  that  is.  occurring  under  con¬ 
ditions  of  heightened  intuitive  awareness. 
Then  there  those  that  occur  in  the  Second 
Domain,  the  preparation  for  the  combat,  and 
these  activities  are  accomplished  in  a  more 
methodical  evaluative  framework.  Decision 
Support  and  the  analytical  manipulation  of 
Information  are  activities  that  are  more 
readily  applicable  to  the  measured  framework 
of  the  planning  processes. 

While  its  data  correlation  and 
management  features  are  fully  operant  in 
the  First  Domain,  the  NCCS(A  )  is  focused 
primarily  on  the  preparation  for  the  battle 
and  hence  is  a  Second  Domain  tool.  In  the 
Second  Domain,  the  preparatory  activities 
include  the  development  of  a  Plan  of  Action, 
and  the  supervision  and  modification  of 
Force  disposition.  It  is  the  Plan  of  Action 
and  the  disposition  of  the  Force  that  serve 
as  the  basis  for  improvisation  during  the 
actual  combat.  Thus  it  is  the  NCCS(A) 
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products  that  lay  down  the  operating  base¬ 
line  upon  which  victory  or  defeat  will  be 
achieved.  Using  the  new  software  tools  of 
vulnerability  assessment,  propagation 
estimation,  counter-surveillance. 

Indications  and  Warning,  and  battle  damage 
assessment  provided  by  NCOS (A),  the  accuracy 
and  timeliness  of  Second  Domain  planning 
and  supervision  (i.e.,  the  control  of  assets 
and  their  disposition)  can  be  improved. 

Additionally,  increases  in  the  range 
of  weapons  has  resulted  in  the  expansion  of 
the  area  within  which  engagement  can  occur. 
The  relatively  unchanged  radius  of  the 
organic  Force  sensor  envelope  does  not 
extend  to  the  effective  range  of  the  newer 
weapons.  This  results  in  an  area  wherein 
the  newer  weapons  provide  a  kill  capability 
while  existing  Force  sensors  provide  no 
surveillance  mechanisms  suitable  for  tar¬ 
geting.  This  set  of  circumstances  places 
an  increased  emphasis  on  the  integration 
and  use  of  information  from  nonorganic 
sensors  and  National  assets.  The  corre¬ 
lation  of  this  additional  information  with 
that  of  organic  sources  has  increased  the 
information  processing  requirements  during 
the  Second  Domain.  It  should  be  noted  at 
this  point  that  the  term  "Information"  (as 
opposed  to  "data")  implies  intelligibility 
and  therefore  a  certain  amount  of  human 
interpretation.  Accordingly,  there  is  a 
new  dependency  on  data  correlation  and 
information  management  within  the  Composite 
Warfare  Commander's  (CWC)  team.  NCCS{A) 
supplies  the  tools  with  which  these  data 
will  be  correlated  and  interpreted  for 
display  as  "Information". 

Finally,  since  the  Commander's 
dependence  on  National  intelligence  infor¬ 
mation.  platform  sensor  data,  and  message 
communications  is  heightened,  he  must  also 
protect  the  electromagnetic  envelope  of  his 
forces  as  thoroughly  as  he  protects  the 
physical  integrity  of  his  ships.  Comple¬ 
mentary  features  of  the  NCCS(A)  tool  also 
provide  him  the  means  for  controlling  this 
"Electromagnetic  Battle".  NCCS{A)  is,  in 
reality,  a  fully  integrated  EW  Decision 
Support  System  whose  analysis  modules  are 
pertinent  to  spectrum  management  as  well  as 
to  information  fusion.  These  interactive 
processes  provide  EW  vulnerability  and 
countermeasures  analysis,  readiness 
assessment  and  asset  control. 

Thus,  the  NCCS(A)  product  as  defined 
by  the  current  PDW  162  procurement  strategy 
will  accomplish  all  of  the  information 
management  and  decision  support  functions 
required  by  the  Electronic  Warfare  Co¬ 
ordinator.  It  will  accommodate  all  of  the 
functions  of  Planning,  Assessment  and 
Supervision  inherent  in  the  management  of 
the  electromagnetic  battle  during  the 
Second  Domain.  It  will  provide  the  basic 
algorithmic  support  for  data  correlation 
and  interpretation  across  the 
organic : non-organic  and  Genser:SCI 
components  in  both  Domains.  Further,  it 
will  decentralize  the  performance  of  these 
tasks  through  the  use  of  a  suite  of 
standardized,  integrated  Advanced  Color 
Workstations . 

Because  its  MMI  is  performed  on  8086 
based  NDI  equipment  (i.e.,  off-the-shelf 
items),  additional  features  that  are  unique 
to  each  warfare  area  specialty  and  to  each 


subordinate  Staff  specialist  can  be  provided 
through  transportable  software  programs. 

The  product  being  created  under  this 
development  initiative,  will  allow  analysis 
packages  that  support  any  of  the  CWC  watch- 
standers  to  be  loaded  and  applied  at  any 
station  within  the  NCCS(A)  equipped  spaces. 
With  its  integrated,  standardized  suite  of 
NDI  equipment  hosting  transportable, 
modular,  processing  software,  the  NCCS(A) 
is  truly  thi  feWC ' s  battle  management  tool  of 
the  90's.  It  provides  a  solid  basis  for  the 
measured  introduction  of  Decision  Support 
and  Artificial  Intelligence  Technology  into 
the  tools  of  tactical  direction  and  battle 
management  that  are  emerging  in  the  Fleet. 

SYSTEM  ARCHITECTURE 

The  functional,  logical  and  physical 
architecture  of  the  NCCS(A)  suite  are 
consistent  through  all  of  its  development 
phases.  As  will  be  seen  in  the  discussion 
of  the  procurement  strategy  for  the  proto¬ 
type,  interim,  and  "full-up"  versions  of 
NCCS(A).  the  following  architectural  designs 
form  the  basis  for  all  aspects  of  its 
evolutionary  development. 

Functional  Architecture 

Functionally,  the  NCCS(A)  system  is  a 
Group  Decision  Support  system  that  is 
composed  of  a  number  of  Advanced  Color 
Workstations  connected  by  a  multi-level 
security,  dual  ring  bus.  These  work¬ 
stations,  though  standardized  in  respect  to 
I/O,  display  and  Operating  system,  contain 
sufficient  CPU  and  memory  to  act  as 
individual  information  processing  systems. 
Accordingly,  the  NCCS(A)  functions  are 
distributed  throughout  a  network  of  inter¬ 
changeable  workstations  served  by  a 
centralized  Relational  Database  and 
Communications  facility.  These  latter 
components  are  hosted  in  a  32  bit,  parallel 
processing  environment  whose  principle 
responsibility  is  to  provide  data 
correlation  and  track  management  support. 

Figure  1,  The  EWCM  DSS  Architecture, 
presents  the  functional  architecture  of  a 
single  decision  support  component  of  the 
NCCS(A)  as  it  appears  to  the  User  of  a 
standard  workstation  -  e.g.,  the  Electronic 
Warfare  Coordinator  (EWC).  The  EWC  has 
facilities  for  processing  his  Electronic 
Battle  Management  analysis  functions  from 
his  Battle  Station,  or  from  any  of  the 
spaces  served  by  the  NCCS(A)  LAN  (i.e.,  its 
dual  ring  bus).  On  it  he  can  iterate 
different  solutions  to  EW  problems  by 
invoking  differing  mathematical  algorithms 
and  analysis  modules  as  decision  support 
tools.  Other  than  data  retrieval  and  down¬ 
loading  the  Common  Tactical  Picture,  these 
analyses  can  be  completed  without  burdening 
the  rest  of  the  NCCS(A)  system. 

Invoking  various  algorithms  and  models 
in  a  coherent,  integrated  manner  requires  a 
Model  Base  Management  System  of  some  kind. 
Addressing  the  implementation  of  this 
function.  Sen  and  Biswas  have  stated:  "... 
at  the  strategic  planning  level  there  is  a 
great  need  for  different  models  to  analyze 
and  support  decisions.  These  models  need 
to  be  integrated  automatically. . .  Inte¬ 
gration  at  the  physical  level ...  requires 
that  the  users  have  a  lot  of  knowledge 
about  implementation  details.  However, 
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Figure  1 

The  EWCM  DSS  Architecture 


integration  of  models  st  the  logical  level 
using  knowledge-based  systems  techniques 
should  make  the  package  independent  of  the 
particular  system  [e.g.,  workstation]  they 
are  implemented  on"  (1).  However,  while 
Sen  recommends  Expert  Systems  techniques 
for  concatenating  the  various  analysis 
models,  a  simpler  approach  using  linear 
modeling  has  been  proposed  by  Pieng-Laing 
(2).  Undoubtedly  a  synthesis  of  these 
approaches  will  be  applied  in  the  mature 
version  of  NCCS(A),  In  the  prototype 
version,  at  least,  various  amounts  of 
operator  intervention  will  be  required. 

Accordingly,  the  functional  archi¬ 
tecture  depicted  in  Figure  1  illustrates 
that  by  downloading  appropriate  applications 
modules  into  the  Model  Base  Management 
System  of  his  processor,  and  through 
distributed  access  to  his  partition  in  the 
Relational  Database  Management  System,  the 
EWC  can  invoke  Rules  Models  and  Decision 
Support  "blackboard"  functions  that  are 
unique  to  his  analysis  responsibilities. 
Standard  text  and  graphics  display,  word 
processing,  and  intra-/extra-NCCS (A) 
Communications  modules  complete  the 
functional  architecture  of  the  workstation. 
Complex  processes  such  as  track  management, 
data  correlation  and  information  fusion 
that  are  common  and/or  too  demanding  for 
the  workstations  are  processed  centrally 
and  the  pertinent  results  provided  to  Users 
via  the  centralized  Database  Manager, 

Logical  Architecture 

^  The  logical  architecture  of  the  NCCS(A) 
IS  illustrated  by  its  treatment  of  battle 
management  information  as  it  is  created 
from  the  correlation  of  data  from  organic 
and  non-organic  sources.  Figure  2,  The 
EWC's  Information  Preparation:  Organic  & 
Non-Organic  Fusion,  presents  five  levels  of 
information  fusion  that  culminate  in  its 
incorporation  in  the  Planning,  Assessment 
and  Supervision  functions  inherent  in 
Second  Domain  Battle  Force  control: 

Levels  one  and  two  portray  the 
collection  of  data  and  its  correlation  as 


similar  source  tracks  that  are  disseminated 
on  standard  NTDS  communications  media. 

Level  3  represents  the  fusion  of  these  data 
in  the  NCCS(A)  correlation  function  where 
dissimilar  source  tracks  and  parametric 
data  are  fused.  At  level  four,  the  multi¬ 
level  security  feature  of  the  dual  ring  bus 
(i.e.,  LAN)  facilitates  the  further 
integration  of  information  across  security 
compartments  within  the  NCOS (A)  suite. 

Here  GENSER  and  SCI  tracks  are  fused  with 
polished  information  from  national  sources. 

The  final  stage  of  NCCS(A)  information 
management  is  the  application  of  the  fused 
information  product  in  the  Battle  Management 
process  as  its  Common  Tactical  Picture, 

This  stage  is  its  use  by  various  members  of 
the  CWC  Team  (in  this  illustration  the  EWC), 
operating  at  their  respective' workstations . 
as  an  adjunct  to  their  own  analysis 
algorithms  and  task  specific  data 
facilities  described  above. 

Physical  Architecture 

The  physical  connectivity  of  the  system 
is  illustrated  in  Figure  3,  the:  "CV  NCCS 
Afloat  Network" .  The  NCCS(A)  strategy  for 
providing  Group  Decision  Support  to  the  CWC 
team  is  accomplished  by  distributing  its 
standardized  workstations  throughout  CVIC. 
SUPPLOT.  TFCC.  and  CIC.  The  "Mainframe" 
processors  are  centralized  in  the  CVIC  space 
and.  as  has  been  noted,  provide  centralized 
database  management ,  communications,  data 
correlation,  and  information  integration 
support. ^  The  Large  Screen  Displays  in  the 
three  principle  CWC  work  areas  facilitate 
briefing,  coordinated  analysis,  and  the 
formation  of  group  consensus.  Consensus 
formation  is  enhanced  because  the 
standardized  display  function  of  the  work¬ 
stations  includes  a  "windows"  capability. 

When  this  ability  to  split  the  screen  for 
the  display  of  results  from  several  tasks 
is  invoked  on  the  Large  Screen  Display  it 
will  enable  the  simultaneous  presentation 
of  planning  considerations  from  several 
(overlapping)  warfare  areas.  This  capa¬ 
bility  is  one  manifestation  of  the  NDI 
management  tools  that  will  be  provided  by 
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The  EWC's  Information  Preparation:  Organic  &  Non-Organic  Fusion 


Figure  3 

CV  NCOS  Afloat  Network 


the  Group  Decision  Support  component  of 
NCOS (A) . 

The  distributed  components  of  the 
system  are  connected  by  a  dual  ring,  multi¬ 
security-level  bus  that  operates  under  a 
Token  control  protocol.  Two  fibre-optic 
transmission  paths  connect  all  components 
of  like  security.  Each  device  is  connected 
to  the  dual  bus  through  a  LAN  Interface 
Unit  (LIU)  which  includes  a  bypass  device 
for  each  ring.  Each  bypass  has  the  capa¬ 
bility  to  act  as  a  switching  node  and  thus 
each  device  is  logically  connected  to  both 
rings.  The  bypass  capability  also  allows 
multiple  redundant  paths  of  transmission  in 
the  event  of  device  failure  or  cable  inter¬ 
ruption.  Accordingly,  in  the  event  of  the 
simultaneous  interruption  of  both  cables  at 
the  same  node  (i.e.,  the  destruction  of  a 
compartment)  the  remaining  nodes  are  auto¬ 
matically  reconstituted  into  a  single  ring. 


Viewed  from  the  perspective  of  security- 
levels.  a  set  of  four  fibre-optic 
transmission  paths  exist  wherein  two  GENSP]R 
rings  and  two  SCI  rings  provide  physically 
separate,  mechanically  redundant  data 
exchange  paths.  Cryptologic  interface 
devices  couple  the  dissimilar  security  links 
and  thereby  provide  a  certified,  sanitized 
path  of  information  between  SCI  sources  and 
users  in  GENSER  spaces.  External  communi¬ 
cations  are  supported  by  connections  to 
either  of  the  segregated  rings  and  a  Battle 
Override  is  provided  whereby  one  work¬ 
station  and  Large  Screen  Display  in  TFCC 
can  be  manually  connected  to  the  SCI  bus. 

COMPONENT  MODULES 

Addressing  the  tools  of  decision¬ 
making,  VAdra.  J.  Boyes  (3)  once  noted: 

"...  two  human  levels  of  [Navy]  decision¬ 
making  must  be  separated  . . .  the  commander 
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. . .  [and]  staff  support  to  senior 
operational  decision  makers."  The  Navy's 
emerging  NCCS(A)  support  for  the  Composite 
Warfare  Concept  of  battle  force  management 
recognizes  that  concept  -  particularly  so 
in  the  case  of  EW  data  and  Information 
fusion.  In  the  management  of  EW  data,  as 
it  flows  from  the  collector  (sensor)  to  the 
senior  commander  (User),  raw  data  is  trans¬ 
formed  into  information  in  several  stages 
of  command  activity. 

The  Composite  Warfare  Commander,  as 
flag  officer  in  tactical  command  is 
supported  by  other  specialized  commanders. 
The  air,  surface,  subsurface  and  strike 
warfare  commanders  command  "by  negation" 
the  various  elements  of  combat  within  their 
purview.  The  CWC  reviews  the  overall 
evolution  of  the  battle  through  the  medium 
of  his  specialized  staff.  The  EW  staff 
officer,  the  Electronic  Warfare  Coordinator 
(EWC) ,  has  responsibility  for  all  EW  and 
C3CM  activities  across  all  of  the  warfare 
areas.  He  is  responsible  to  them  and  to 
the  CWC  for  all  EW  and  C3CM  plans  and 
doctrine  in  support  of  the  mission. 

Active  and  passive  sensors  aboard  the 
various  force  platforms  collect  data  on 
adversary  activities.  These  data  represent 
statistically  independent  events  and  are 
analyzed  and  interpreted  at  the  mid¬ 
management  level  by  each  platform's  EW 
Supervisor.  These  synthesized  elements  of 
"meta-information"  ate  then  incorporated 
into  the  NTDS(ACDS)  component  of  the  TSS. 
Through  this  medium  they  become  available 
to  the  NCCS(A)  information  management  suite 
where  they  are  integrated  and  fused  with 
intelligence  data  from  national  sources. 

Using  the  resulting  high  confidence 
information,  staff  officers  assess  Force 
plans  and  activities  and  recommend  appropri¬ 
ate  modifications  to  the  CWC.  This  section 
discusses  the  major  units  of  information 
analysis  in  the  NCCS(A)  suite.  These  are 
the  Afloat  Correlation  sub-System  (ACS),  the 
Electronic  Coordinator's  Module  sub-System 
(EWCM)  and  the  decision  support  components 
of  each.  EWCM,  being  created  within 
NCCS(A),  is  the  first  of  a  family  of 
decision  support  tools  that  are  being 
developed  for  use  by  the  CWC  staff  analyst/ 
specialists.  All  of  these  decision  tools 
will  overlay  the  Common  Tactical  Picture 
presented  to  the  Battle  Force  Commander  by 
its  ACS  component. 


Electronic  Warfare  Coordinator's  Module 


The  EWCM  subsystem  will  provide  auto¬ 
mated  plans  development,  monitoring  and 
supervision  to  the.  EWC.  It  will  also  allow 
continual  re-evaluation  of  plan  suitability 
and  assessment  of  alternative  actions. 

When  the  EWCM  sub-system  is  fully  integrated 
into  the  NCCS(A)  information  management 
suite,  it  will  accept  interpreted  parametric 
data  from  platform  sensors  and  combat 
direction  systems.  It  will  receive  force 
status  reports,  equipment  availability  and 
transmission  schedules,  intelligence 
estimates  and  atmospheric  measurements. 

The  EWCM  partition  of  the  database  will 
house  Force  Operations  Orders.  Rules  of 
Engagement.  Ocder(s)  of  Battle  and 
intelligence  data. 

Using  EWCM,  the  EWC  can  evaluate  the 
composite  meaning  of  the  various  parameters 
of  organic  and  non-organic  electromagnetic 
data.  He  can  estimate  the  effects  of 
existing  atmospheric  and  environmental 
conditions  on  the  propagation  character¬ 
istics  of  both  his  own.  and  his  adversary's 
EW  and  C3CM  devices.  By  analysis,  the  EWC 
will  be  able  to  investigate  the  probable 
effects  of  committing  some  of  his  weaponry 
without  the  need  to  retain  a  great  deal  of 
simulated  data  in  his  own  mind.  He  will 
not,  of  course,  burden  the  C2  systems  with 
these  trial  solutions.  At  the  same  time, 
he  will  be  able  to  visually  observe  the 
impact  of  his  prospective  decisions  as  over¬ 
lays  on  the  Common  Tactical  Picture  and  as 
quantitative  Utility  values  on  the  Decision 
Blackboard . 


Afloat  Correlation  System 


The  centralized,  parallel  processing 
components  of  NCCS(A)  which  deal  with  data 
correlation  and  information  fusion  comprise 
the  ACS  subsystem.  The  workstation  analysis 
modules  can  call  upon  these  components  for 
the  results  of  Organic  and  Non-Organic  Track 
Correlation,  and  for  Collection  Management, 
Indications  and  Warning,  and  Targeting 
Support,  Figure  4:  "Data  Fusion  Domain", 
prepared  by  the  JDL  Data  Fusion  Subpanel, 
presents  a  taxonomy  of  fusion  activities  as 
they  relate  to  the  logical  functions  of 
Track  Management. 
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Data  Fusion  Domain 
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The  ACS  will  approximate  these 
functions  at  levels  1  and  3.  The  track 
correlation  function  of  ACS  is  centralized 
in  its  parallel  processing  mainframe. 

Threat  assessment,  intelligence  assessment 
support,  collection  management,  and 
Indications  and  Warning  generation  are 
decentralized  functions  that  are 
distributed  across  the  NCCS(A)  suite. 

Table  1,  The  Functional  Distribution  of  the 
ACS  Sub-System  Activities,  presents  the 
allocation  of  ACS  tasks  between  the 
centralized  processing  and  the  workstation 
facilities . 

When  considering  the  centralization- 
decentralization  of  NCCS(A)  elements,  it 
will  be  remembered  that  there  are  several 
aspects  of  these  elements  which  can  be 
controlled  or  distributed.  Beside  the 
relative  distribution  of  actual 
computations  (i.e.,  processes)  the 
standardization  of  workstations  provides 
the  potential  for  distributing  and  thus 
decentralizing  logical  (i.e.,  operational) 
functions,  intellectual  (i.e.,  analytical) 
activities,  and  authentication  (i.e., 
certifying)  responsibilities.  Thus,  the 
ACS  sub-system,  as  incorporated  in  the 
integrated  NCCS(A)  design  will  centralize 
processor  intensive  data  function  and  track 
correlation  processes  and  distribute  the 
management  activities  of  situation 
assessment,  planning,  and  control  among 
the  contributing  workstations  of  the  CWC 
staff  team. 

DECISION  SUPPORT  MECHANISMS 

One  of  the  most  forward  looking 
features  of  the  NCCS(A)  system  is  the 
incorporation  of  its  decision  support 
components.  A  Decision  Support  System  (DSS) 
is  a  man-machine  couple  that  facilitates 
the  incorporation  of  experience  and  instinct 


in  decision-making.  It  enhances  the  process 
of  choosing  by  using  data  recall,  sophis¬ 
ticated  data  manipulation  and  graphic 
display  to  stimulate  managerial  evaluation 
of  a  set  of  choices.  Decision  Support 
systems  are  unique  among  computer  based 
information  systems  in  their  ability  to 
apply  ad-hoc  simulation  as  a  medium  for 
hypothication  (What  If?)  and  automated  goal 
seeking  to  the  solution  of  problems. 

The  dichotomy  of  combat  decision¬ 
making  identified  by  VAdm.  Boyes  is  well 
served  by  providing  these  capabilities  at 
the  CWC  level.  Table  2,  The  Characteristics 
of  Hierarchical  Decision  Making,  is  based  on 
Simon's  pioneering  work  in  organizational 
behavior  (4).  It  compares  the  levels  of 
management  in  a  Battle  Force  with  the 
intentions  (Activity  Or ientation) ,  data,  and 
decision  processes  they  entail  and  with  the 
automated  tools  which  support  them.  It 
illustrates  that  at  the  Flag  level,  as  the 
decision  process  becomes  less  structured, 
the  criteria  upon  which  the  choices  are 
based  are  best  served  by  the  Decision 
Support  System  technology  that  is  emerging 
in  contemporary  Information  Systems 
applications . 

The  definition  of  structured  problems 
provided  by  Jones  offers  an  interesting 
counterpoise  with  which  to  delimit  un¬ 
structured  decisions:  "A  structured  problem 
is  one  . . . [for  which]  decision  rules  can  be 
specified  that  allow  for  problem  definition, 
designation  of  alternatives,  and  selection 
of  the  best  alternative"  (5).  Clearly,  this 
exposition  illustrates  the  dichotomy  of  the 
NCCS(A)  problem  solving  dilemma.  On  the 
one  hand,  the  system  itself  is  a  decentral¬ 
ized  group  consensus  forming  tool  serving 
the  CWC  team.  On  the  other,  it  is  both  a 
calculating  tool  and  a  personal  decision 
support  system.  It  not  only  calculates 


Table  1 

The  Functional  Distribution  of  the  ACS  Sub-System  Activities 


Functional 

Category  Specific  Functions  Distribution 


*  Accept  Sensor  reports 

Sensor  *  Accept  data  linked  remote  sensor  data  Centralized 
1/0  *  Provide  common  time/nav  reference 

*  Monitor  status  of  sensors 


*  Coordinate  multiple  sensor  searches 

*  Cue  and  handoff  sensor-sensor  data 

*  Coordinate  emitter  muting 

Sensor  *  Prioritize  sensor  activities  Centralized 

Control  *  schedule  sensor  events,  modes 

*  provide  threat  alerts:  hostile  contacts 

*  Report  data  base  (data,  history,  quality) 

*  Accept  sensor  cues 


*  Associate  data  from  all  sources 

*  Maintain  temporal  target  trades 

*  Combine  associated  feature  data  to; 

*  Improve  location  accuracy 

*  Remove  identification  ambiguity 

Data  *  Increase  confidence  in  data  Centralized 

Fusion  •  Estimate  temporal  activity  of 

periodic  emitters  or  emitter  complexes 

*  Emitter  kinematic  tracks 

*  Emitter  activity  history 


*  Monitor  all-source  contact  data  base 

*  Detections,  tracks,  files,  events 

*  History  file 

*  Aggregate  multiple  targets  into 
threat  complexes 

Situation  *  Develop  order  of  battle  assessment  Decentralized 

Assessment  •  Prioritize  threats  and  opportunities 

*  Monitor  ownship  emissions  for  emcon 
compl iance 

*  Maintain  platf orm/attr ibute  data  base 

*  Provide  survivability  assessment 
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The  Characteristics  of  Hierarchical  Decision  Making 


yANAGEUENT 

ACTIVITY 


,  ACITVITY 
DRIENTATION 


DATA 


DECISION 


OECISIOM 

SUPPORT 

TOOL 


FUQ  PLAMNINfi 
TOP  MAKAOEMENT 


ACCOMPLISH 
•0  MISSION 


SOURCES  FORMAT  CURRENCY  SITUATION 


EXTERNAL  OVERVIEW  HISTORICAL  UNIQUE 


PROCESS  CRITERIA 


NON- 

STRUCTURED  INTUITIVE 


OSS 


PLATFORM 
PUNNING  AND 
MANAGEMENT 


ALLOCATE  ANO 

EMPLOY 

RESOURCES 


MIXED  PLATFORM  MIXED 
SOURCES  SPECIFIC 


C3 

SYSTEM 


WEAPONS  AND 
SENSORS  OP 
CONTROL 


EXECUTE  INTERNAL  SYSTEM  REAL 

MISSION  SOURCES  SPECIFIC  TIME 

OBJECTIVES 


RECURRING  STRUCTURED  OPTOtftZE 


WEAPONS 


SYSTEM 


track  and  data  correlation  probabilities 
but  it  also  allows  the  individual  decision 
maker  access  to  algorithms  for  solving  the 
structured  components  of  his  warfare 
problems.  Then,  the  individual  structured 
problems  having  been  solved,  in  the  aggre¬ 
gate  (through  the  Blackboard  technique)  it 
becomes  a  personal  decision  support  system 
at  the  workstation  level.  Finally, 
combining  the  CWC  Staff  activities  on  the 
individual  workstations  into  the  comprehen¬ 
sive  NCCS(A)  system,  the  group  consensus 
itself  becomes  the  decision  support  tool 
that  serves  the  CWC  in  the  solution  of  his 
own  unstructured  problem(s). 

Decision  Modules 

The  EWCM  sub-system  component  of 
NCCS(A)  will  include  components  that  house 
the  following  existing  decision  modules: 
Vulnerability  Assessment  Device  (VAD) , 
Integrated  Refractive  Effects  Prediction 
System  (IREPS),  the  HF  prediction  system  - 
PROPHET,  Communication  Plan  Automation 
System  (CPAS),  and  a  radio  frequency  de- 
confliction  algorithm.  Ideally,  the  full- 
up  version  of  NCCS(A)  will  be  constructed 
to  allow  these,  and  any  algorithms  that  are 
created  in  the  future,  to  be  embedded  with¬ 
out  modification.  It  is  unlikely  that  the 
interim  EWCM  will  reach  this  level  of 
sophistication  but  the  concept  of  insert- 
able  applications  modules  is  certainly  a 
candidate  for  P3I  and  is  being  considered 
in  the  ongoing  detailed  definition. 

Model  Base  Control  Functions 

The  EWCM  is  designed  to  allow  the 
operator  to  make  iterative  judgements  and 
incremental  evaluations  of  the  effects  of 
decisions.  Accordingly,  a  key  feature  of 
the  model  base  management  system  is  its 
capability  to  format  and  transpose  data 
elements  from  one  application  module  for 
use  within  another.  This  concept  not  only 
provides  the  logical  integration  that  will 
unburden  the  User  but  it  will  also  ensure 
the  capability  of  EWCM  to  expand  its  model 
base  as  increasingly  sophisticated  analysis 
algorithms  become  available.  It  guarantees 
that  the  completed  NCCS(A)  system  will  be  a 
receptive  host  to  the  analysis  packages 
that  will  ultimately  be  developed  by  the 
other  CWC  staff  specialties. 


Temporary  recall  and  replay  of 
previously  considered  simulation  scenarios 
will  also  be  required  of  the  model  control 
function.  A  scratchpad  will  be  allocated 
for  the  storage  of  several  iterations  of  an 
ongoing  analysis.  Parametric  changes  will 
be  logged  and  available  with  synthesized 
alternative  plan  elements  temporarily  saved. 
This  feature  will  enable  the  User  to  review 
the  process  by  which  he  has  arrived  at  his 
latest  proposed  solution  to  every 
unstructured  decision. 

The  Blackboard  Device 

The  present  NCCS(A)  application  of  DSS 
technology  is  primarily  concerned  with  the 
active  and  passive  use  of  the  electro¬ 
magnetic  spectrum  through  the  creation  (and 
modification)  of  spectrum  use  plans  and 
frequency  allocations.  Notwithstanding  the 
rigors  of  its  creation  during  the  early 
part  of  the  Second  Domain,  the  process  of 
maintaining  changes  to  the  plan  for 
frequency  use/surveillance  is  a  process 
that  entails  many  sequential  and  inter¬ 
related  decisions.  Without  a  comprehensive 
blackboard  tool,  under  conditions  of 
imminent  combat  when  the  electronic  battle 
is  at  its  most  intense,  the  correlation  and 
assimilation  of  so  large  a  number  of  dis¬ 
associated  decisions  may  limit  the  utility 
of  the  system.  Accordingly,  a  simplified 
presentation  medium  must  be  developed.  One 
model  that  has  been  proposed  is  an 
adaptation  of  the  techniques  of  Expected 
Value  theory.  This  paradigm  offers  both  a 
memory  aid  and  a  decision  correlation  tool. 
This  concept,  developed  as  a  decision  tree 
and  applied  to  the  concatenated  decisions 
of  spectrum  planning  is  illustrated  in 
Figure  5.  The  Blackboard:  Planning  Tree. 

This  representation  presents  the 
decision-maker  with  an  initial  decision 
recommendation  along  with  the  audit  trail  of 
all  its  precursor  decisions.  With  the  deci¬ 
sion  tree  solution  displayed  on  the  screen, 
the  DSS  user  can  evaluate  the  proposed 
solution  according  to  his  perspective  of  the 
combat  situation  as  a  specialist-member  of 
the  CWC  team.  Following  his  review  of  the 
proposed  optimal  solution  the  user  can  then 
evaluate  all  of  the  intermediate  decisions 
(i.e.,  nodes  of  the  tree)  that  contributed 
to  its  selection.  In  so  doing  he  can  accept 
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EQPT#1 


EQPT#2 

EQPT#3 


each  machine  generated  solution  or  he  can 
modify  any  of  the  elements  of  that  solution. 
For  example,  the  decision  matrix  of  a  fre¬ 
quency  assignment  paradigm  could  be  reviewed 
and  a  subjective  weighting  of  payoffs^ 
assigned  according  to  the  operator's  judge¬ 
ment.  These  weighs  can  be  combined  with 
the  Expected  Values  of  the  decision  node. 
This  process,  in  effect,  transmutes  the 
mathematically  structured  stochastic 
approach  of  Decision  Theory  into  a  solution 
that  encompasses  the  behaviorally  cognizant 
elements  of  Prospect  Theory  (6).  Alterna¬ 
tively,  the  user's  review  of  the  prediction 
mechanisms  might  call  for  an  application  of 
Bayes  theorem  and  he  might  momentarily 
invoke  a  model  of  the  theorem  to  aid  in  his 
adjustment  of  the  probability  estimate. 

The  vehicle  for  this  interactive 
modification  is  the  decision  tree  portrayal 
of  the  problem  set.  Its  manipulation 
provides  a  "blackboard"  that  incorporates 
the  instinctual  and  intuitional  processes 
which  are  the  basis  of  Decision  Support 
technology.  There  has  apparently  been  some 
success  in  applying  a  variation  of  this 
technique  to  the  Group  Decision  process. 
Johnston  reports:  "The  Group  Decision  Aid 
guides  the  decision  making  process  through 
the  selective  evolution  of  a  multi-level 
decision  tree,  consisting  of  possible 
actions  and  events  resulting  from  them  ... 
the  system  asks  the  individual  group 
members  to  input  their  estimates  of  the 
likelihood  as  well  as  the  value  of  specific 
decision  outcomes"  (7). 

APPLICATION 

Decision  Support  is  first  of  all,  a 
man-machine  interaction.  As  has  been  said, 
it  is  a  process  of  data  manipulation  which 
is  assisted  by  automated  tools  and  which 
encourages  the  User  to  reach  a  conclusion 
about  his  unstructured  problem  solution. 
Bennett  succinctly  puts  it:  "A  recurrent 
theme  in  DSS  (Decision  Support  System) 
research  is  User  learning. ...  A  DSS  does 
not  solve  problems;  it  lets  individuals 
exploit  their  own  skills  in  problem  solving. 
The  obvious  strategy  for  DSS  design  is  to 
support  first  and  extend  later"  (8). 
Recursive  development  of  a  problem  solution 
is  implicit  in  the  Decision  Theory  under¬ 
pinnings  of  the  Blackboard  mechanism. 


The  following  example  is  an 
illustration  of  a  hypothetical  man/machine 
interaction  in  electronic  battle  management. 
It  suggests  both  the  approximate  interactive 
decision  play  between  user  and  machine  and 
the  implementation  of  the  Decision  Theory 
concept  underlying  the  Blackboard  device 
discussed  above.  The  scenario  is  presented 
in  terms  of  a  matrix  solution  to  the  series 
of  successive  decision  problems.  These 
problems  could  be  concatenated  however  and 
the  composite  presented  as  shown  in  Figure 
5.  A  "SelectZoom"  effect  (as  illustrated) 
could  be  used  by  the  End-User  to  decompose 
the  problem  into  its  component  elements  for 
selective  treatment  of  its  various  technical 
and/or  operational  aspects.  This  appli¬ 
cation  presents  the  standard  one  shot 
Decision  Theory  technique  to  the  assignment 
of  frequencies.  The  scenario  extends  over 
three  time  periods  for  the  purposes  of 
illustrating  the  impact  of  additional  data 
upon  the  solution; 

"It  has  become  necessary  to  create  a 
net  for  communications  with  several  distant 
platforms  that  are  moving  according  to  a 
predefined  schedule.  The  Electronic  War¬ 
fare  Officer  has  exercised  the  vulnerability 
prediction  algorithms  of  his  DSS  and  has 
determined  that  there  are  two  modes  of 
communication  which  will  fulfill  his 
requirement  within  an  acceptable  level  of 
ECM/ECCM  risk.  He  has  the  option  of 
establishing  a  HF  or  of  using  a  special 
national  channel.  It  is  likely  that 
authorization  for  the  use  of  the  national 
asset  will  not  be  forthcoming  so  will  he 
first  evaluate  the  HF  net  requirements." 

Among  the  interrelated  elements 
involved  in  the  selection  of  an  appropriate 
HF  transmission  frequency  are:  frequency 
availability,  equipment  capability,  broad¬ 
cast  range,  and  power  levels.  Compounding 
the  difficulty  of  the  choice  are  various 
transmission  variables  that  change  according 
to  location  and  time  of  day.  In  this 
illustration  of  DSS  use  of  a  blackboard, 
the  user  intends  to  select  from  among  three 
authorized  frequencies  the  one  that  best 
suits  the  needs  of  the  force  net.  He  has 
algorithms  available  which  provide  range 
characteristics  from  a  propagation 
prediction  model,  environmental  parameters 
from  an  automated  atmospheric  sampling 
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system,  and  an  algorithm  which  addresses 
the  problem  of  harmonic  interference. 

Case  1.  A  set  of  three  frequencies  is 
available  for  the  assignment  to  the  force  HF 
network.  The  net  must  achieve  a  transmis¬ 
sion  to  a  series  of  moving  platforms.  It 
has  been  determined  that  the  solution  payoff 
must  attain  a  figure  of  merit  of  at  least 
300  for  reliable  transmission  to  occur. 
Combinations  of  the  predictions  of  a  series 
of  embedded  stochastic  tables  such  as: 
Surface  Duct  phenomena.  Enhanced  Range 
predictors.  Elevated  Duct  phenomena, 
sunspot  activities,  and  the  output  from  a 
transmission  path  prediction  model  provides 
the  following  estimates  of  the  occurrence 
of  a  skip  effect: 

51  <  300  nautical  miles;  p=.35 

52  >  300  but  <  1000  nautical  miles.  p=.4 

53  >  1000  nautical  miles;  p=.25 

Expected  Distance  of  Transmission  Skip 
Versus  Frequency  Assignment  (case  1) 


State  of 
nature 
(P) 

Frequency 

Assigned 

SI 

.35 

S2 

.4 

S3 

.25 

Expected  Value 
(EV) 

f  1 

230 

185 

365 

246 

f2 

375 

320 

100 

284 

f3 

150 

100 

800 

292  <- 

Note:  Payoff  figures  of  merit  are  assigned 

for  illustration  and  should  be  developed 
through  separate  research  (see:  Composite 
EW  Quantifiers,  Ormsby  (9)) 

The  Expected  Value  (EV)  column  presents 
the  weighted  figure  of  merit  for  each 
frequency.  Note  that  none  of  the  expected 
payoffs  meets  the  required  minimum  figure 
of  merit  of  300.  Using  only  this  "screen" 
the  operator  would  select  frequency  f3  as 
being  the  best  choice  even  though  the 
required  parameters  of  communication  were 
not  met.  The  principle  of  choice  for 
differentiation  between  f3  or  f2  is  not  so 
clearly  evident  however.  Trade-offs 
between  these  two  frequencies  that  are  not 
contained  in  the  present  problem  are  not 
obvious  and  thus  the  operator  is  prevented 
from  considering  them  in  this  decision. 
Accordingly,  in  the  absence  of  any  other 
information,  the  operator  is  reduced  to 
making  the  un-enlightened  selection  of  f3. 

Case  2.  In  addition  to  choosing  a 
frequency,  the  operator  must  select  from 
among  three  power  settings  in  consonance 
with  equipment  performance  profiles,  EMCON 
restrictions,  and  ECM  intercept  predictions. 
The  states  of  nature  and  Figures  of  merit 
are  derived  from  reliability  estimates, 
duty  cycles,  atmospheric  predictions. 

Expected  Performance  Profiles  at 
selected  power  Outputs  (case  2) 


State  of 
nature 
(P) 
Power 
Output 

SI 

.60 

S2 

.15 

S3 

.35 

Expected  Value 
(EV) 

pol 

230 

185 

365 

257 

po2 

375 

320 

100 

298 

po3 

150 

100 

800 

305  <- 

Note:  Payoff  figures  of  merit  are  assigned 

for  illustration  and  should  be  developed 
through  separate  research  (see:  Composite 
EW  Quantifiers,  Ormsby  (9)) 

The  appropriate  choice  here  is  clearly 
po3.  However,  (as  an  elaboration  of  the 
decision  support  concept)  the  operator  may 
have  reservations  about  the  interrelation 
between  the  selection  of  the  frequency  and 
the  power  output  as  they  relate  to  the 
potential  contribution  the  effect  of  the 
skip  made  to  the  Expected  Value  of  the 
figure  of  merit  calculations.  This  could 
result  for  instance,  from  his  past 
experience  in  a  similar  situation  v;hen  the 
predicted  phenomenon  failed  to  occur.  In 
this  case  he  might  choose  ps2  since  it 
almost  met  the  minimal  selection  criteria 
and  it  also  (hypothetically)  satisfies  his 
own  instinctive  evaluation  of  the 
situation.  The  principle  of  choice  here  is 
"Satisficing"  since  the  selected  frequency 
does  not  meet  the  minimum  criteria  but  the 
match  is  close  enough  to  preclude  the 
expenditure  of  more  resources  in  creating 
another  solution. 

Case  3 .  In  the  final  iteration  of  the 
example,  the  operator  consults  the 
"Operational  Events"  and  movement  reports 
to  determine  the  anticipated  positions  of 
the  platform  he  desires  to  have  communi¬ 
cations  with.  He  observes  that  the 
formations  planned  for  the  Force  at  this 
timeline  will  probably  also  invalidate  the 
use  of  f3  which  depends  very  heavily  on  the 
effects  of  the  skip.  While  he  contemplates 
this  line  of  reasoning,  he  consults  equip¬ 
ment  availability  files  to  assess  the 
various  equipment  and  emitter  combinations 
(EE)  available.  He  then  causes  the 
harmonic  effects  prediction  algorithm  to 
evaluate  three  of  the  combinations  in 
relation  to  the  probability  of  interference 
from  platform  reflected  harmonics  (H)  at 
f3,  po2.  Predictions  of  intermodulation 
are  particularly  stringent  in  the  case  of 
f2  and,  as  a  result  of  this  analysis 
(hypothetically)  effectively  reduces  its 
radiated  power.  The  harmonic  effects 
predictor  creates  the  figure  of  merit 
payoffs  shown  below. 


Expected  Harmonic  Interference  Versus 
Equipment-Emitter  Availability  (case  3) 


State  of 

HI 

H2 

H3 

Expected  Value 

nature 

(EV) 

(P). 

.25 

.  60 

.35 

Equipment- 

Emitter 

EEl 

230 

185 

365 

296 

EE2 

130 

180 

350 

263 

EE  3 

150 

100 

800 

305  <- 

Note:  Payoff  figures  of  merit  are  assigned 

for  illustration  and  should  be  developed 
through  separate  research  (see;  Composite 
EW  Quantifiers,  Ormsby  (9)) 

The  user  is  prompted  by  this  analysis 
to  choose  equipment-emitter  combination 
EE3.  At  this  point  the  operator  may  care 
to  revise  his  prediction  about  the 
advisability  of  using  f3,  po2  because  the 
combination  of  the  transmitter  equipment 
and  emitter  device  selected  may  be  unique 
and  therefore  not  covered  by  his  previous 
experience. 
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In  reality,  harmonic  interference  would 
probably  effect  all  of  the  .frequencies  and 
power  setting  possibilities  and  would  call 
for  a  total  re-evaluation  (a  recursion  on 
the  present  example).  In  all  probability, 
additional  factors  would  have  to  be 
considered  as  adjustments  in  power  output 
would  require  reference  to  the  Rules  of 
Engagement,  and  EMCON  Rules;  re-selection 
of  the  radiating  surface  would  incorporate 
EOOB,  Table  of  Equipment,  and  Equipment 
availability  review;  and  frequency  selection 
criteria  might  require  a  further  analysis 
of  platform  location(s).  the  acceptable 
level  of  through-put  reliability,  or  the 
minimum  distance  criteria. 

DEVELOPMENT  STRATEGY 

NCCS{A)  is  an  evolutionary  progression 
of  Battle  Force  Management  tools  that  are 
based  in  the  already  operational  Flag  Data 
Display  System  (FDDS)  component  of  the 
Tactical  Flag  Command  Center  (TFCC).  A 
modular  upgrade  of  the  data  transmission 
facilities  of  the  FDDS  and  the  inclusion  of 
a  network  of  advanced  color  workstations 
will,  in  the  near  term,  result  in  an  early 
operational  capability  for  dissimilar  source 
data  integration  and  for  planning  and  super¬ 
vision.  As  a  precursor  to  this  Interim 
NCCS(A),  an  extensive  validation  of  the 
concept  is  being  conducted  in  an  operational 
environment  aboard  the  USS  Vinson. 

Various  Fleet  End-User  sources  have 
developed  elementary  prototype  devices  that 


model  some  of  the  elements  of  the  NCCS(A) 
system.  By  interconnecting  a  suite  of 
these  prototypes  through  an  Ethernet  bus  a 
simulation  of  the  interrelationship  of  the 
distributed  functions  of  these  components 
can  be  established.  While  it  is 
acknowledged  that  the  aggregate  effect  of 
these  devices  is  relatively  unsophisticated 
in  comparison  with  that  of  the  Interim 
version  (i.e.,  the  FDDS  upgrade),  the  End- 
User  reception  and  use  of  these  devices  is 
being  measu^d  as  an  indication  of  the 
applicability 'of  functions  of  the  more 
sophisticated  Interim  system. 

The  final  phase  of  the  NCOS (A) 
development,  its  "full-up"  version  will 
complete  the  TFCC  upgrade  cycle.  It  will 
incorporate  the  full  set  of  advanced  propa¬ 
gation  assessment  algorithms  hosted  in  the 
model  base  management  system.  The  latter, 
probably  served  by  a  graph  based  integration 
mechanism,  will  enhance  the  construction  of 
ad-hoc  models  of  spectral  behavior.  This 
assessment  feature  will  allow  pertinent 
decision  parameters  to  be  individually 
analyzed  and  incorporated  into  an  event 
sensitive  battle  management  solution.  Using 
the  Decision  Theory  concepts  found 
acceptable  during  the  precursor  develop¬ 
ments,  state-of-the-art  consensus  forming 
protocols  will  be  embedded  in  this  CWC 
Decision  Support  System  that  will  allow 
group  collaboration  in  the  comprehensive 
management  of  the  Force  Energy  Spectrum 
across  all  of  the  composite  warfare  domains. 
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BELIEF  REPRESENTATION  FOR  FUSION  USING  EVIDENCE  THEORY 


John  W.  Betz 


The  Analytic  Sciences  Corporation 
55  Walkers  Brook  Drive 
Reading,  MA  01867 


Evidence  theory  is  especially  appropriate  for  the  fusion  of  data  from  multiple  sources 
because  of  its  ability  to  accommodate  uncertainty  and  conflict,  A  limitation  of  this  power¬ 
ful  formalism  in  fusion,  however,  has  been  the  inability  to  rigorously  assign  belief  meas¬ 
ures  to  assertions  based  on  numerical  outputs  from  sensors.  A  rigorous  approach  to 
representing  belief  in  numerical  quantities  is  presented  that  extends  the  applicability  ot 
evidence  theory  in  data  fusion. 

This  approach  applies  to  assertions  that  a  numerical  quantity  takes  on  values  in  an 
interval  or  on  a  half-line,  extending  Dempster-Shafer  theory  to  symbolic  information  derived 
from  sensor  data.  The  method  assumes  that  incomplete  statistical  models  can  be  developed 
for  the  numerical  quantity  in  the  form  of  a  closed  set  of  .  probabi lity  density  functions, 
even  when  a  single  density  function  cannot  be  assigned  because  the  underlying  statistics  ot 
the  data  are  not  known  perfectly  and  the  effects  of  both  the  sensor  and  the  processing 
cannot  be  modeled  perfectly.  The  representation  explicitly  accounts  for  simplifications  in 
analytical  models,  and  for  unknown  parameters  that  affect  the  statistical  description. 

The  derivation  of  Shafer  belief  functions  from  the  set  of  probability  density  functions  is 
described  based  on  the  use  of  mass  density  functions,  introduced  by  Strat  to  represent  the 
assignment  of  mass  to  assertions  based  on  numerical  quantities.  The  contribution  of  this 
paper  is  the  development  of  a  rigorous,  statistically-based,  procedure  for  deriving  the  mass 
density  function.  The  necessary  and  sufficient  conditions  for  the  set  of  probability  den¬ 
sity  functions  to  produce  a  Shafer  belief  function  are  presented,  and  it  is  shown  that  the 
lower  distribution  function  originally  described  by  Dempster  both  is  a  lower  bound  on  the 
belief  function  and  has  the  properties  of  a  belief  function.  A  simple  expression  for  the 
plausibility  function  in  terms  of  the  upper  and  lower  distribution  functions  is  also  de¬ 
rived. 

The  exposition  provides  additional  conceptual  and  mathematical  linkage  between  evidence 
theory  and  Bayesian  theory.  In  particular,  when  the  set  of  density  functions  has  unity 
cardinality,  the  resulting  mass  density  function  is  equivalent  to  a  probability  density 
function  and  the  belief  function  is  Bayesian.  Consequently,  the  variation  among  members  ot 
the  set  of  density  functions  can  be  viewed  as  an  expression  of  ignorance  in  evidence  theory. 
The  analysis  thus  reinforces  the  view  of  Bayesian  statistics  as  a  limiting  case  of  evidence 
theory . 
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INTRODUCTION 

Evidence  Theory,  also  known  as  Dempster- 
Shafer  Theory,  is  especially  appropriate  for 
the  fusion  of  data  from  multiple  sources  be¬ 
cause  of  its  ability  to  accommodate  uncer¬ 
tainty  and  conflict.  -  The  belief  measures 
that  represent  uncertainty  in  Evidence  The¬ 
ory  are  both  less  constraining  and  richer 
than  probabilistic  representations,  since 
uncertainty  resulting  from  ignorance  and 
statistical  variability  can  be  separately 
represented  numerically,  without  the  limita¬ 
tions  of  a  single  point  probability  repre¬ 
sentation.  Furthermore,  mass  may  be 
assigned  not  only  to  the  atomic  propositions 
that  are  the  mutually  exclusive,  exhaustive 
propositions  in  the  frame  of  discernment, 
but  also  to  sets  of  atomic  propositions. 
The  ability  to  assign  mass  directly  to  sets 
of  propositions  facilitates  the  fusion  of 
data  from  different  sensors  that  provide 
qualitatively  different  information  at  dif¬ 
ferent  levels.  Dempster's  Rule  provides  a 
consistent  mechanism  for  combining  evidence 
from  multiple  independent  sources,  even  with 
vastly  different  qualities  of  evidence  and 
with  partial  conflict.  Under  certain  common 
assumptions.  Evidence  Theory  may  be  consid¬ 
ered  as  a  generalization  of  Bayesian  statis¬ 
tics,  so  that  when  Bayesian  representation 
of  uncertainty  can  be  provided,  Evidence 
Theory  provides  Bayesian  results. 

The  restriction  of  Shafer's  development 
of  Evidence  Theory  to  discrete  propositions 
limits  the  ability  to  represent  belief  in 
numerical  evidence  for  applications  involv¬ 
ing  the  use  of  sensor  data,  such  as  data 
fusion.  Not  only  has  a  rigorous,  automated 
mechanism  been  unavailable  for  deriving  or 
representing  the  belief  in  numerical  quanti¬ 
ties  or  in  propositions  derived  from  those 


quantities,  but  a  representation  is  lacking 
that  constitutes  a  natural  framework  for  hu¬ 
man  experts  to  express  their  heuristic  be¬ 
lief.  A  rigorous  approach  to  representing 
belief  in  numerical  quantities  is  presented 
to  address  these  deficiencies.  This  ap¬ 
proach  has  many  desirable  attributes,  in¬ 
cluding  the  possibility  of  automation, 
providing  a  natural  representation  for  heu¬ 
ristic  belief,  and  contributing  to  the  link 
between  Evidence  Theory  and  statistics. 

The  second  section  of  this  paper  reviews 
relevant  aspects  of  Evidence  Theory,  includ¬ 
ing  upper  and  lower  probabilities,  proper¬ 
ties  of  support  and  plausibility  functions, 
and  mass  density  functions  for  representing 
belief  in  numerical  quantities.  New  results 
are  contained  in  the  third  section,  which 
shows  how  statistical  descriptions  of  nu¬ 
merical  quantities  may,  under  certain  con¬ 
straints,  be  represented  as  Evidence  Theory 
belief  measures.  The  fourth  and  final  sec¬ 
tion  summarizes  the  results  of  this  paper. 

THEORETICAL  FOUNDATIONS 

Evidence  Theory  is  based  on  statistical 
developments  by  Dempster,  extended  by  Shafer 
to  form  a  non-Bayesian  theory  that  distin¬ 
guishes  between  objective  and  subjective  in¬ 
formation  in  considering  chance  events. 
More  recently,  Strat  developed  the  concept 
of  a  mass  density  function  to  represent  be¬ 
lief  in  some  types  of  numerical  information. 
The  results  in  the  third  section  are  based 
on  these  theoretical  foundations. 

Upper  and  Lower  Probabilities 

Upper  and  lower  probabilities  are  intro¬ 
duced  by  Dempster  in  Ref.  l  to  describe  the 
effect  of  a  multivalued  mapping  between  ran¬ 
dom  quantities.  This  concept,  its  extension 
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to  conditional  probabilities,  and  its  appli¬ 
cation  to  the  combination  of  independent 
sources  of  information,  form  the  basis  for 
Shafer's  work  and  the  resulting  mathematical 
theory  of  evidence. 

The  concept  of  upper  and  lower  probabili¬ 
ties  is  based  on  a  multivalued  mapping,  T, 
between  a  pair  of  spaces  X  and  W,  so  that  T 
assigns  a  subset  Tx  c  W  to  every  x  in  X. 
The  problem  is  to  characterize  the  probabil¬ 
ity  associated  with  w,  a  subset  of  W,  in 
terms  of  a  probability  measure,  \l,  defined 
on  X.  When  T  is  single- valued ,  the  prob¬ 
ability  associated  with  w  can  usually  be  ex¬ 
pressed  directly  in  terms  of  |li.  When  F  is  a 
set -valued  function,  however,  only  upper  and 
lower  probabilities  can  be  defined  over  sub¬ 
sets  of  W. 

For  any  w  c  W,  define  w*  as  the  subset  of 
X  containing  all  values  of  x  that  are 
mapped  into  any  set  in  W  that  intersects  w, 
that  is, 

w*=  {xeX,  Fx  n w  0}  ( 1 ) 

where  0  is  the  null  set.  Similarly,  define 
w*  as  the  subset  of  X  containing  all  values 
of  X  that  are  mapped  into  any  non-null  sub¬ 
set  of  w,  that  is, 

w*={xeX,Fx  5*^  0,Fx  C  w}  (2) 

Consequently,  W*  equals  W* ,  and  is  the  do¬ 
main  of  F.  For  any  set  w  for  which  the 
probability  measure  is  defined  on  w  and 
w*,  the  upper  probability  of  w,  p*(w),  is 
defined  as 


p*(w)  =jw(w*)//i(W*) 

(3) 

and  the  lower  probability  of  w 

,  P*(w), 

,  is 

defined  as 

P*  (w)  =  )///(W* ) 

(4) 

When  X  is  a  random  variable  and 

W  is  a 

sub- 

set  of  the  set  of  real  numbers 

,  upper 

and 

lower  distribution  functions. 

F*(V) 

and 

F*(v),  can  be  defined  for  w  by 

F*  (v)  =  p*  (w  <  v) 

(5) 

F*  (v)  =  p*  (w  <  v) 

(6) 

where  {w  <  r}  denotes  the  event  {w  :  v  <  v,  Vvew} . 
Neither  the  lower  distribution  function  nor 
the  upper  distribution  function,  however, 
bounds  the  probability  of  the  random  vari¬ 
able,  V,  in  a  finite  interval.  For  a  and  b 
finite  with  a<b,  p(a<v<b)  may  be  smaller 
than  F*(b)-F*(a)  and  larger  than 
F’(b)-F,(a) . 

Evidence  Theory 

In  contrast  to  the  statistical  orienta¬ 
tion  of  Dempster's  work,  Shafer  presents  in 
Ref.  2  an  integrated  theory  of  evidence 
based  on  a  philosophical  foundation  that 
differs  from  Bayesian  probability.  This 
original  exposition  is  supplemented  by  sev¬ 


eral  more  recent  descriptions,  defenses,  and 
expansions  of  Evidence  Theory  in  Ref.  3  and 
Ref.  4.  There  are  also  many  useful  summa¬ 
ries  in  the  literature,  for  example.  Ref.  5 
and  Ref.  6,  augmented  by  attempts  to  view 
Evidence  Theory  from  a  statistical  view¬ 
point,  as  in  Ref.  7. 

Shafer's  developments  provide  a  set  of 
semantics,  functions,  and  operations  in  a 
consistent  mathematical  framework  that  is 
structurea  to  avoid  the  inability  of 
Bayesian  measures  to  distinguish  between  ob¬ 
jective  variability,  or  chance,  and  subjec¬ 
tive  variability,  or  ignorance.  An  interval 
representation  of  belief  in  a  proposition  is 
employed  using  an  ordered  pair  of  nonnega¬ 
tive  numbers,  each  no  greater  than  unity. 
The  ensuing  discussion  denotes  these  numbers 
the  support  and  the  plausibility,  deviating 
slightly  from  Shafer's  semantics.  The  sup¬ 
port  function  indicates  how  much  the  avail¬ 
able  evidence  supports  the  proposition  to 
the  exclusion  of  any  other  proposition, 
while  the  plausibility  indicates  how  much 
the  evidence  does  not  refute  the  proposi¬ 
tion.  Denoting  the  support  for  proposition 
A  by  S(A),  the  plausibility  by  P(A)  ,  and  the 
complement  of  A  by  A, 

P(A)  =  1-S(A)  (7) 


and 


S(A)  <  P(A)  (8) 

When  S(A)  is  equal  to  P(A),  the  belief  is 
Bayesian  and  has  all  the  standard  properties 
of  probabilities.  The  difference  between 
plausibility  and  support  is  termed  the  igno¬ 
rance,  and  indicates  the  inability  to  commit 
to  either  the  proposition  or  its  complement. 

All  propositions  of  interest  are  subsets 
of  the  frame  of  discernment,  which  is 
equivalent  to  a  probabilistic  sample  space. 
In  contrast  to  probability  theory,  mass  rep¬ 
resenting  belief  can  be  assigned  to  any  ele¬ 
ment  of  the  power  set  of  the  frame  of 
discernment,  not  just  the  exhaustive,  mutu¬ 
ally  exclusive  propositions,  if  a  unit  of 
mass  is  distributed  among  the  propositions, 
the  support  for  a  proposition  consists  of 
the  mass  attributed  to  that  proposition  and 
to  its  subsets,  while  the  plausibility  is 
the  support  plus  all  the  mass  attributed  to 
supersets  of  the  proposition. 

A  support  function  is  defined  in  Ref.  2 
as  any  function  that  has  the  following  prop¬ 
erties  on  a  frame  of  discernment  0  , 

S(0)  =  O  (9) 


${0}  =  1 


(10) 


SCAiUAjU  .  .  .  UA„)  a  X  Ai) 

IC{1,2 . n) 

I  <p 

(11) 

when  the  {A^}  are  subsets  of  0,  and  for 
every  positive  integer  n,  where  |i|  is  the 
cardinality  of  I.  The  property  in  Eq.  9  is 
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called  superadditivity,  and  is  essential  to 
the  development  in  the  following  section.  A 
plausibility  function  is  defined  by  the  com¬ 
bination  of  Eq.  7  and  the  properties  of  a 
support  function. 

Mass  Density  Functions 


or 


b  00 

P(a,  b)  =  I  I  M(x,  y)dydx  = 

-  oc  max(a.  X) 


00  niin(b,  y) 

I  J  M(x,y)dxdy 


a  -  00 


(15) 


The  development  of  Evidence  Theory  has 
emphasized  finite  frames  of  discernment. 
Numerical  quantities  on  the  set  of  real  num¬ 
bers  or  some  infinite  subset  clearly  violate 
this  assumption.  Mass  density  functions, 
introduced  as  mass  functions  in  Ref.  8,  pro¬ 
vide  an  elegant  mechanism  for  representing 
and  manipulating  belief  that  a  continuous 
quantity  takes  on  a  value  within  specific 
intervals  of  the  set  of  real  numbers.  Two- 
dimensional  mass  density  functions,  the  fo¬ 
cus  of  the  present  paper,  represent  the  mass 
assigned  on  a  frame  of  discernment  that  con¬ 
sists  of  a  half-open  interval  or  half-line 
on  the  real  number  line,  or  the  set  of  real 
numbers  itself.  The  set  of  focal  proposi¬ 
tions,  or  propositions  to  which  mass  may  be 
assigned,  is  constrained  for  two-dimensional 
mass  density  functions  to  those  of  the  form 

w  =  {ve  (a,  b]}  (12) 

Mass  assignments  to  sets  of  non-cont iguous 
intervals  can  be  represented  by  mass  density 
functions  of  high  dimension.  In  many  appli- 
cations  involving  the  use  of  numerical  data, 
however,  interest  is  confined  to  proposi¬ 
tions  involving  numerical  values  occupying 
contiguous  intervals,  and  a  two-dimensional 
mass  density  function  is  adequate. 


where  the  region  of  integration  is  also  in¬ 
dicated  in  Fig.  1.  The  support  for  the  un¬ 
ion  of  the  two  disjoint  intervals  is  equal 
to  the  sum  of  the  supports  for  each  inter¬ 
val,  consistent  with  the  constraint  of  a 
two-dimensional  mass  density  function.  Eq. 
13  is  used  as  the  basis  for  describing  and 
deriving  mass  density  functions,  to  build  on 
Shafer's  extensive  analysis  of  the  proper¬ 
ties  of  support  functions. 

When  the  partial  derivatives  of  the  sup¬ 
port  function  with  respect  to  the  interval 
bounds  are  defined,  the  mass  density  func¬ 
tion  can  be  expressed  directly  from  Eq.  13, 

M(x,y)=-^S(x,y)  (18) 

Because  the  support  function  is  bounded,  all 
constant  terms  and  terms  that  are  functions 
only  of  either  x  or  y  in  the  mass  density 
function  are  zero. 

The  definition  of  mass  density  functions 
in  Eq.  13,  combined  with  the  properties  in 
Eqs.  9,  10,  and  11,  yields  two  important 
properties  of  mass  density  functions:  nor- 
sjBalization  and  nonnegativity.  Normalization 
states  that  a  mass  density  function  has  unit 
volume , 


A  two-dimensional  mass  density  function 
M(x,y)  is  any  nonnegative  function  with  unit 
volume  that  is  zero  on  the  half-plane  y  <  x. 
This  is  consistent  with  the  defined  rela¬ 
tionship  between  a  Shafer  support  function 
and  a  mass  density  function,  where  the  sup¬ 
port  function  for  the  proposition  in  Eq.  12 
is  formally  denoted  S[{ve(a,b]}]  and  abbre¬ 
viated  S(a,b).  A  mass  density  function, 
M(x,y),  is  defined  formally  by  the  integral 
relationship  from  Ref.  8, 

b  b  by 

S(a,  b)  =  J  J  M(x,  y)dydx  =  J  J  M(x,  y)dxdy  (13) 

ax  a  a 

Eq.  12  combines  with  Eq .  13  to  imply  that 

the  mass  density  function  is  zero  on  the 
half-plane  y  <  x.  Both  the  region  of  sup¬ 
port  for  a  mass  density  function  and  the 
region  of  integration  for  computing  the  sup¬ 
port  function  are  shown  in  Fig.  l.  Concen¬ 
trating  the  mass  near  the  hypotenuse  in  Fig. 
1  assigns  mass  to  narrow  intervals,  indicat¬ 
ing  the  ability  to  describe  accurately  the 
behavior  of  the  numerical  quantity.  Mass 
assigned  near  the  upper  left-hand  corner  in¬ 
dicates  inability  to  describe  detailed  be¬ 
havior.  From  Eq.  7  and  Eq.  13,  the 
plausibility  function  expressed  in  terms  of 
the  mass  density  function,  is 

P(a,b)=  1-S(-  <»,a)--S(b,  oo)  (14) 


I  I  M(x,  y)dydx  =  1  (17) 

-  «  X 

This  property  follows  from  Eq.  10  and  the 
fact  that  the  frame  of  discernment  is  either 
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the  interval  (-00,®)  oi*  some  subinterval,  so 
that  $(-»,«)  equals  unity. 

Nonnegativity  of  the  mass  density  func¬ 
tion, 


M(x,y)  >  0,Vx,y  (18) 

follows  from  the  superadditivity  of  Shafer's 
support  function  in  Eq.  11.  Ref.  9  shows 
that  superadditivity  of  the  support  function 
is  equivalent  to  nonnegativity  of  the  mass 
density  functions . 

Dy:RTVATIQN  OF  BELIEF  IN  NUMERICAL  QUANTITIES 

The  elegant  representation  of  belief  in 
continuous  quantities  by  mass  density  func¬ 
tions  tends  to  be  a  very  unnatural  way  for 
algorithms  or  human  experts  to  directly  ex¬ 
press  belief.  while  Ref.  8  discusses  rela¬ 
tionships  between  possibility  functions  in 
fuzzy  set  theory  and  the  derivation  of  mass 
density  functions,  this  approach  may  be  no 
more  natural  for  representing  the  belief  in 
sensor  data.  Sensor  performance  is  typi¬ 
cally  described  statistically,  and  a  mecha¬ 
nism  for  relating  statistical  descriptions 
to  mass  density  functions  is  expected  to  be 
most  natural  and  useful.  When  there  is  com¬ 
plete  knowledge  of  the  statistical  behavior 
of  the  numerical  quantity,  there  is  no  igno¬ 
rance  of  the  quantity's  behavior,  and  the 
random  behavior  can  be  described  by  a  prob¬ 
ability  density  function.  However,  when 
there  is  a  lack  of  knowledge  of  the  statis¬ 
tical  behavior,  one  way  to  express  the 
available  knowledge  is  to  provide  a  set  of 
probability  density  functions  whose  elements 
each  represent  possible  random  behav^ior  of 
the  quantity.  For  example,  different  sta¬ 
tistical  characterizations  may  describe  sen¬ 
sor  data  acquired  under  different 
conditions.  The  conditions  associated  with 
a  particular  set  of  sensor  data  may  not  be 
known,  and  estimating  the  information  from 
the  data  may  be  impossible  or  impractical. 
Under  certain  conditions,  the  expression  of 
uncertainty  as  a  set  of  probability  density 
functions  can  be  used  to  derive  a  mass  den¬ 
sity  function. 

Fundamental  Result 

Consider  a  set  V  of  random  variables 
characterized  by  different  probability  den¬ 
sity  functions,  where  is  the  set  of  prob¬ 
ability  density  functions  that  describe  all 
the  random  variables  in  V.  Each  random 
variable  in  V  can  also  be  described  by  a 
different  single-valued  mapping  from  a  ran¬ 
dom  variable  y  that  is  uniformly  distributed 
on  (0,1],  and  T'  is  defined  to  be  the  set  of 
these  mappings.  If  x  is  the  set  of  all 
subintervals,  x,  of  (0,1],  F  is  a  mapping  of 
X  defined  by 


Fx  =  {F'y ,  yex  ^  X}  (19) 


and  if  W  is  the  set  of  all  subintervals  of 
the  range  of  F' ,  then  F  is  a  multivalued 
mapping  between  X  and  W  as  discussed  in 
Ref.  1.  The  probability  measure  on  X  is 
defined  as  the  probability  that  y  is  in  x, 
so  the  probability  of  the  domain  of  F  is 
unity.  The  set  of  probabilities  assigned  to 
any  w  in  W,  where  w  is  defined  in  Eq.  12,  is 
composed  of  the^  probabilities  that  v  is  in 
(a,b]  for  every  v  in  V.  The  probability  of 
w  can  be  bounded  by  the  upper  and  lower 
probabilities  defined  in  Ref.  l.  Using  Eq. 
2  and  Eq.  4,  the  lower  probability  for  w 
equal  to  (a,b]  in  W  is 

P*  (w)  =  p({y^* ;  w*  =  {x  :  a  <  Fx  <  b,  xeX}})  (20) 

where  for  x  equal  to  any  (a^^t)^]  in  X,  the 
probability  measure  is 


p(yex)  =  bo  -  Bo 


(21) 


Typically,  the  set  is  composed  of  a 
countable  number  of  probability  density 
function  parametric  forms  {fi(v;ai)},  each  with 
a  possibly  uncountable  number  of  parameter 
variations  icLi) .  where  each  element  fly  of  the 
vector  Oi  describes  a  different  probability 
density  function  parameter,  and  is  bounded 
by  the  interval  Under  conditions 
to  be  defined,  the  set  can  be  used  to 
derive  the  support  and  plausibility  for  an 
interval  w,  based  on  Theorem  A. 2  in  Ref.  10. 
The  theorem  states  that  when  a  closed  convex 
set  of  probability  functions  is  defined  over 
the  frame  of  discernment,  and  for  every  set 
of  propositions  on  the  frame  of  discernment. 


Z 

I  C  (1.2. 


(-  l)l‘l^^  minrpCn  Aj)! 

n)  L  iel  J 


(22) 


when  the  minimum  is  taken  over  the  set  of 
probability  functions,  then  for  every  propo¬ 
sition  A  in  the  power  set  of  the  frame  of 
discernment , 


S(A)  =  min[p(A)]  (23) 

In  the  application  of  this  theorem  to  the 
representation  of  belief  in  continuous  quan¬ 
tities,  interest  is  confined  to  propositions 
relating  to  contiguous  intervals.  Conse¬ 
quently,  the  number  of  elements  in  the  frame 
of  discernment,  n,  in  Eq.  22,  can  be  re¬ 
stricted  to  two.  Moreover,  the  requirement 
that  the  set  of  probability  functions  be 
closed  and  convex  is  sufficient  and  not  nec¬ 
essary.  This  theorem  holds  whenever  there 
exists  a  unique  minimum  probability  for 
every  set  of  propositions  on  the  frame  of 
discernment . 

Application  of  this  theorem  to  elements 
of  the  set  W  provides  the  fundamental  re¬ 
sult:  When  V  is  a  continuous  quantity  whose 
statistics  are  described  by  some  probability 
density  function  in  the  set  ff,  if  for  all 
a<b<c<d  in  the  frame  of  discernment, 
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(24) 
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f  f(v)dv 
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f(v)dv 
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J 

where  the  min[] 

operator  selects 

the  prob- 

ability  density 

function  in 

9  that 

minimizes 

the  definite  integral,  then  for  all  inter- 

vals  (a,b]  on  the  frame  of 

discernment , 

S(a,b) 

=  min 

r  J  wdv 

u(b  -  a) 

(25) 

fe? 

L-* 

a 

where  the  unit 

step 

function  u(*) 

indicates 

zero  support  when  the  lower  limit  of  the 
interval,  a,  is  greater  than  the  upper  limit 
of  the  interval,  b.  While  pathological 
cases  can  be  developed  to  show  that  Eq.  24 
does  not  always  hold,  most  practical  sets  of 
probability  density  functions  satisfy  Eq. 
24. 

Denote  by  fj^(v;a,b)  the  probability  den- 

b 

sity  function  in  ?  that  minimizes  |  f(v)dv. 

a 

Although  is  a  function  of  a  and  b,  it 

may  vary  only  with  v  over  some  range  of  in¬ 
tervals,  so  that  within  this  range  of  inter¬ 
vals  , 

b 

S(a, b)  =  |fL(v; a,b)  ii(b  - a)dv  (26) 

a 

As  the  interval  changes,  however,  a  differ¬ 
ent  probability  density  function  in  ?  may 
apply,  and  the  transition  between  probabil¬ 
ity  density  functions  may  be  either  smooth, 
as  in  the  case  of  smoothly  varying  parameter 
values  within  a  single  parametric  form,  or 
discontinuous,  with  a  sudden  change  between 
parameter  values  or  parametric  forms. 

When  the  partial  derivative  of  fj^(v;x,y) 
is  defined  with  respect  to  y,  and^  Eq.  24 
holds,  the  mass  density  function  that  corre¬ 
sponds  to  S(x,y)  is  obtained  by  equating  the 
integrands  in  Eq .  21  and  Eq .  26  and  differ¬ 

entiating  with  respect  to  b,  then  substitut¬ 
ing  X  and  y  for  a  and  b, 

M(x,  y)  =  -^fL(x;  x,  y)  u(y  -  x)  +  IlCx;  x,  x)6{x  -  y) 

^  (27) 

The  mass  density  function  is  zero  wherever 
the  same  probability  density  function  is 
used  to  calculate  support  in  Eq.  25,  except 
along  the  line  x  equal  to  y,  where  the  mass 
density  function  is  a  ridge  with  value  equal 
to  the  minimum  value  of  the  probability  den¬ 
sity  functions  in 

When  there  is  no  ignorance  about  the  sta¬ 
tistical  characteristics  of  the  quantity  v, 
the  set  ?  consists  of  a  single  element. 


f(x),  and  the  mass  density  function's  volume 
is  concentrated  on  the  line  x  equal  to  y, 

M(x,y)  =  f(x)6(x-y)  (28) 

Equation  13  and  Eq.  15  show  that,  in  this 
case,  the  support  equals  the  plausibility 
for  all  intervals,  and  the  result  is  a  con¬ 
ventional  statistical  description.  When 
there  is  ignorance  in  the  statistical  de¬ 
scription  3^,  and  Eq.  24  holds,  Eq.  25  and 
Eq.  27  provide  the  means  to  compute  mass 
density  functions  from  statistical  descrip¬ 
tions,  Using  the  equivalence  of  mass  den¬ 
sity  function  nonnegativity  and  support 
function  superadditivity,  however,  Eq.  24 
need  not  be  verified  separately.  Rather, 
the  mass  density  function  can  be  developed 
without  regard  for  Eq.  24,  and  if  the  re¬ 
sulting  mass  density  function  is  nonnega¬ 
tive,  Eq.  24  is  guaranteed  to  hold. 

Bounds  on  Support  and  Plausibility 

Dempster's  upper  and  lower  distributions 
provide  bounds  and  alternative  expressions 
for  the  support  and  plausibility.  A  lower 
bound  on  the  support  function,  S_(a,b) ,  can 
be  derived  from  the  upper  and  lower  distri¬ 
bution  functions  in  Eq.  5  and  Eq.  6, 

S_(a,b)  =  max[0,  (F*(b)  -  F  *  (a))]  (29) 

It  is  proven  in  Ref.  9  that 

SJa,b)  <  S(a,b)  (30) 

Moreover,  it  is  also  shown  that  S_(a,b)  sat¬ 
isfies  the  properties  of  a  support  function 
expressed  in  Eqs .  9,  10,  and  11,  and  thus 

can  be  used  in  replacement  of  the  exact  sup¬ 
port  function  at  the  cost  of  adding  some 
ignorance.  A  corresponding  pseudo-mass  den¬ 
sity  function  can  be  derived  from  S_(a,b), 
using  the  relationships  defined  in  Eq.  13 
and  Eq.  16.  As  shown  in  Ref.  9,  the  pseudo¬ 
mass  density  function  retains  all  the  prop¬ 
erties  of  a  mass  density  function. 
Interestingly,  using  the  pseudo-mass  density 
function  in  Eq.  15  yields  the  plausibility, 
P(a,b),  rather  than  an  upper  bound  on  the 
plausibility.  Hence,  the  plausibility  can 
be  easily  calculated  from  the  upper  and 
lower  distribution  functions 

P(a,b)  =  F*(b)-F*(a)  (31) 

Example 

To  illustrate  the  derivation  of  evidence 
theory  belief  from  incomplete  statistical 
descriptions  in  a  data  fusion  context,  con¬ 
sider  two  sensor  platforms  monitoring  traf¬ 
fic  on  a  highway.  A  patrol  car  equipped 
with  a  speed  measuring  radar  estimates  the 
speed  of  each  vehicle,  and  an  aircraft  using 
an  optical  device  monitors  traffic,  estimat¬ 
ing  vehicle  speed  with  a  timing  device  and 
distance  markings  on  the  pavement.  Fusion 
of  reports  occurs  at  a  central  station  in 
radio  contact  with  the  two  platforms,  using 
the  incomplete  data  provided.  In  order  to 
maintain  the  emphasis  on  representation  of 
sensor  data,  the  problem  of  associating  ve- 
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hides  reported  by  the  different  platforms 
is  avoided  by  assuming  sparse  traffic.  Fur¬ 
thermore,  the  mechanics  of  the  fusion  opera¬ 
tions  are  de~emphasized  relative  to  the 
primary  emphasis  of  representing  belief  in 
sensor  data  to  support  fusion. 

The  example  is  designed  to  illustrate  the 
differences  between  fusion  of  numerical  in¬ 
formation  before  extraction  of  symbolic  in¬ 
formation,  and  the  fusion  of  symbolic 
information  separately  inferred  from  the  in¬ 
dividual  information  sources.  The  fusion  of 
numerical  information  before  extraction  of 
symbolic  information  is  not  always  possible, 
as  shown  in  the  example,  but  when  it  is  pos¬ 
sible,  more  information  is  preserved  after 
fusion. 

Suppose  the  specifications  for  the  radar 
indicate  that  the  random  errors  in  estimat¬ 
ing  speed  are  additive,  uniformly  distrib¬ 
uted  within  +  4  miles  per  hour  of  the 
calibrated  speed,  and  independent  of  the  ac¬ 
tual  speed.  The  radar  calibration  procedure 
is  guaranteed  to  place  the  mean  measurement 
within  ±  2  miles  per  hour  of  the  actual 
speed,  with  the  calibration  bias  of  the  par¬ 
ticular  unit  unknown.  Furthermore,  errors 
associated  with  manually  estimating  vehicle 
speed  from  the  aircraft  using  the  highway 
markings  and  the  timing  device  are  modeled 
as  additive,  independent  of  actual  vehicle 
speed  at  highway  speeds,  and  dominated  by 
the  reflexes  of  the  human.  Suppose  experi¬ 
ments  show  that  these  errors  are  also  uni¬ 
formly  distributed,  with  constant  mean  of 
zero  but  extent  that  varies  from  ±  2  mph  to 
i  5  mph,  depending  on  the  alertness  of  the 
human . 

If  the  speed  estimate  from  the  patrol  car 
radar  is  reported  as  57  mph,  the  true  speed 
can  be  modeled  by  a  set  of  probability  den¬ 
sity  functions, 

9^1  -  |f(v)  :  f(v)  =  i  [u(v  ~  a  -  4)  -  u(v  -  a  +  4)],  55  <  a  <  59| 

(32) 

while  when  the  speed  estimate  from  the  air¬ 
craft  is  reported  as  53  mph,  the  true  speed 
can  be  modeled  by  a  set  of  probability  den¬ 
sity  functions, 

^2=|f(v)  :  f(v)  =-^[u(v-53-a)-u(v-53  +  a)],2  <  a  <  5 j 

(33) 

The  associated  mass  density  functions,  sup¬ 
port  functions,  and  plausibility  functions 
for  the  true  speed,  based  on  each  individual 
sensor  report  and  derived  from  the  statisti¬ 
cal  descriptions  outlined  in  the  example  de¬ 
scription,  are  shown  in  Fig.  2  and  Fig.  3 
respectively,  based  on  derivations  in  Ref. 
9.  Figure  2  shows  that  there  is  zero  sup¬ 
port  for  the  proposition  that  the  speed  is 
between  51  mph  and  53  mph,  since  con¬ 
tains  probability  density  functions  that  as¬ 
sign  zero  probability  to  this  event. 


Conversely,  plausibility  of  0.25  assigned  to 
this  event  reflects  the  fact  that  the  larg¬ 
est  probability  assigned  to  this  event  by 
any  probability  density  function  in  is 
0.25.  Propositions  involving  larger  inter¬ 
vals  and  those  near  the  center  of  the  inter¬ 
val  containing  oc  tend  to  have  larger 
support.  Fusion  of  this  information  using 
Dempster's  Rule  for  mass  density  functions 
presented  in  Ref.  8  yields  a  single  mass 
density  function  that  expresses  the  combined 


INTERVAL  LOWER  LIMIT,  MPH 


(a)  Mass  Density  Function 


INTERVAL  LOWER  LIMIT,  MPH 


(b)  Support  Function 


INTERVAL  LOWER  LIMIT,  MPH 
(c)  Plausibility  Function 

Figure  2  Belief  Representations  for 
Speed  Estimation  from 
Radar 
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sitions  concerning  the  speed  of  the  vehicle 
occupying  an  interval . 

When  multiple  sensors  provide  information 
on  different  frames  of  discernment,  fusing 
the  numerical  information  may  be  more  diffi¬ 
cult.  For  example,  the  sensor  might  provide 
information  in  different  coordinate  frames, 
in  different  units  of  measure,  or  at  differ¬ 
ent  levels  of  detail.  One  approach  is  to 
define  a  frame  of  discernment  on  the  product 
space  of  the  multiple  sensors.  Manipulating 
mass  density  functions  on  a  product  space 
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INTERVAL  LOWER  LIMIT,  MPH ! 


(a)  Mass  Density  Function 


INTERVAL  LOWER  LIMIT,  MPH 


(b)  Support  Function 


INTERVAL  LOWER  LIMIT,  MPH 
(c)  Plausibility  Function 

Figure  3  Belief  Representations  for 
Speed  Estimation  from 
Aircraft 


frame  of  discernment,  however,  can  be  pro¬ 
hibitively  complicated.  Another  alternative 
is  to  transform  the  measurements  from  the 
multiple  sensors  onto  a  common  fusion  do¬ 
main,  so  that  the  information  may  be  com¬ 
bined  in  a  common  frame  of  discernment. 
This  transformation  process  may  also  be  com¬ 
plicated,  and  may  also  introduce  additional 
uncertainty Another  alternative  is  to 
coarsen  the  frames  of  discernment  to  a  com¬ 
mon  symbolic  frame,  as  described  in  Ref.  2. 
The  information  can  easily  be  combined  on 
the  symbolic  frame  using  Dempster's  Rule. 
In  the  highway  surveillance  example,  the  ob¬ 
server  at  the  central  station  may  be  inter¬ 
ested  in  the  type  of  vehicle,  with  the  frame 
of  discernment  consisting  of  car,  van,  and 
truck.  If  the  central  station  receives  in¬ 
formation  from  the  patrol  car  that  the  vehi¬ 
cle  has  a  commercial  license  plate  that  is 
known  never  to  be  assigned  to  cars,  this 
information  assigns  a  unit  of  mass  to  the 
set  {van,  truck}.  The  airborne  observer  may 
be  able  to  estimate  the  length  of  the  vehi¬ 
cle  with  an  optical  sensor  at  known  alti¬ 
tude,  with  vehicle  length  serving  as  a 
discriminant  between  the  three  vehicle 
types.  As  for  the  speed  estimates  previ¬ 
ously  described,  suitable  models  of  the 
length  estimation  process  can  be  used  to  ob¬ 
tain  a  mass  density  function  for  the  length 
estimate . 

The  separate  patrol  car  and  the  airborne 
observer  reports  providing  information  on 
vehicle  type  cannot  be  fused  directly,  since 
the  observations  apply  to  fundamentally  dif¬ 
ferent  frames  of  discernment.  Both  observa¬ 
tions  can  be  fused  on  the  frame  of 
discernment  describing  vehicle  type,  using 
known  mappings  from  license  plate  type  and 
vehicle  length  to  vehicle  type.  This  map¬ 
ping  of  length  to  vehicle  type  is,  in  ef¬ 
fect,  a  coarsening  of  the  mass  density 
function  to  a  discrete,  symbolic  frame  of 
discernment.  When  this  coarsening  of  a  mass 
density  function  to  a  discrete  frame  of  dis¬ 
cernment  is  performed  at  individual  sensors, 
the  fusion  process  is  simplified,  at  the 
cost  of  losing  flexibility  and  detail  since 
the  frame  of  discernment  must  be  predefined. 

DISCUSSION 

A  rigorously  justified  method  for  deriv¬ 
ing  evidence  theory  belief  descriptions  from 
statistical  models  of  sensor  performance  has 
been  presented.  The  method  is  based  on  the 
use  of  mass  density  functions  that  restrict 
the  set  of  focal  propositions  concerning  nu¬ 
merical  values  to  of  contiguous  intervals. 
The  two-dimensional  mass  density  functions 
emphasized  in  the  paper,  which  represent 
mass  assignments  to  single  contiguous  inter¬ 
vals,  are  adequate  for  many  sensor  fusion 
applications.  This  approach  has  many  at¬ 
tributes,  including: 


Formally  linking  evidence 
theory  and  statistical 
theory  representations  of 
uncertainty 
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•  Providing  a  rigorous  way 
to  derive  belief  from 
statistical  models 

•  Offering  an  alternative, 
more  intuitive  way  for 
experts  to  describe  un¬ 
certainty  in  the  sensor 
outputs  when  rigorous 
derivations  are  not  used. 

The  method  is  based  on  the  seminal  work  of 
Dempster,  integrated  with  Shafer's  emphasis 
on  the  properties  of  belief  functions,  and 
Strat's  elegant  representation  of  mass  as¬ 
signments  to  numerical  quantities. 

A  simple  example  shows  how  both  low-level 
fusion  of  numerical  information  and  extrac¬ 
tion  of  belief  in  symbolic  information  to 
support  high-level  fusion  are  supported  by 
this  approach.  Low-level  fusion  preserves 
mass  assignments  over  the  entire  original 
frame  of  discernment,  allowing  more  flexi¬ 
bility  in  extracting  symbolic  information 
from  the  fusion  product.  High-level  fusion 
of  symbolic  information  extracted  from  the 
measurements  of  individual  sensors  is 
equivalent  to  fusion  on  coarsenings  of  the 
frames  of  discernment  associated  with  the 
original  mass  density  functions.  When  two 
sensors  provide  inherently  different  infor¬ 
mation,  for  example  in  different  coordinate 
systems,  or  when  one  source  of  information 
is  inherently  symbolic,  the  disparate  types 
of  information  from  the  different  sensors 
must  be  mapped  to  a  common  frame  of  discern¬ 
ment  before  fusion.  The  mapping  is  gener¬ 
ally  simplest  when  the  frame  of  discernment 
used  for  fusion  involves  discrete,  symbolic 
propositions . 

This  approach  to  deriving  belief  in  nu¬ 
merical  quantities  represents  a  contribution 
towards  the  fusion  of  numerical  information 
using  Evidence  Theory,  but  not  a  completed 
solution  to  the  practical  problems  of  repre¬ 
senting  belief  in  numerical  quantities. 
While  providing  a  mechanism  for  translating 
incomplete  statistical  descriptions  into  be¬ 
lief  measures,  it  does  not  address  the  deri¬ 
vation  of  the  statistical  descriptions.  In 
some  cases,  as  in  the  examples  presented, 
these  is  a  natural  and  straightforward  way 
to  acquire  the  requisite  statistical  infor¬ 
mation.  In  some  other  applications,  there 
is  considerably  more  difficulty  in  deriving 
a  set  of  density  functions  to  describe  the 
numerical  quantity  of  interest.  in  the 
cases  when  analytical  formulations  may  need 
to  be  supplemented  by  heuristics,  expression 
of  heuristic  information  in  terms  of  prob¬ 
ability  density  functions  may  be  a  more 
natural  and  richer  representation  for  sensor 
experts,  rather  than  forcing  experts  who 


generally  think  in  terms  of  statistical  mod¬ 
els  to  describe  their  belief  directly  in 
terms  of  mass  assignments  on  a  continuous  or 
discrete  frame  of  discernment.  Therefore, 
this  approach  has  applications  for  both  rig¬ 
orous  and  heuristic  representations  of  be¬ 
lief  in  numerical  quantities. 
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ABSTRACT 

•rae'SjSfiwTOJiMV 

.  Post-event  diagnosis  requires  multiple  information  requests  and  takes  time.  Processing  is  done  in  spurts  based  on 
requests. 

.  The  proactive  method  saves  critical  steps  and  time  but  requires  continuous  processing  effort  and  additional  resources. 

.  M..rh  of  the  teouired  rrrocessing  was  already  being  done  within  NORAD,  but  under  post-event,  high-stress  conditions; 
McSiS  wS  n«  adequately  orgluiized  or  shared  among  the  key  NORAD  information  centers. 

.  The  different  centers  within  NORAD  must  learn  to  anticipate  each  others  information  needs  as  critical  situations  develop. 


*  Study  sponsored  by  U.S.  Air  Force,  Armstrong  Aerospace  Medical  Research  Laboratory,  Wnght-Patterson 
Air  Force  Base,  Ohio  under  subcontract  to  Systems  Development  Corporauon. 
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INTRODUCTION  AND  SUMMARY 


In  1985,  the  Air  Force  Armstrong  Aerospace  Medical 
Research  Laboratory  sponsored  a  study  of  NORAD's  Integrated 
Warning  and  Attack  Assessment  (TTW&A)  process.  The  purpose  of 
the  study  was  to  determine  whether  a  proactive  approach  to  the  fus¬ 
ing  of  intelligence  and  other  information  with  sensor  data  could  sig¬ 
nificantly  reduce  the  time  required  to  reach  a  high-confidence 
assessment.  The  results  were  positive,  and  were  briefed  to  repre¬ 
sentatives  of  the  Air  Force  Space  Command, 


ITW&A  DECISIONMAKING  PROCESS 

Responsibility  for  the  ITW&A  decision  process  and  its 
products  lies  with  CINCNORAD.  Other  commands  such  as  the 
NCA  and  CINCSAC  also  involve  key  decisionmakers  to  whom  the 
results  of  the  ITW&A  process  represent  time-critical  inputs.  In 
brief,  CINCNORAD  monitors  the  developing  strategic  situation, 
tries  to  reduce  uncertainty,  and  decides  what  his  report  to  the  other 
commands  will  be.  He  must  also  manage  the  widespread  informa¬ 
tion  system  which  brings  him  the  requisite  information  on  which  to 
base  his  decisions. 

As  a  matter  of  course,  this  information  system  must  provide 
CINCNORAD  with  an  accurate  and  timely  perception  of  the  devel¬ 
oping  situation.  But  it  should  also  help  him  to  "walk  through"  criti¬ 
cal  processing  and  decision  sequences;  to  keep  track  of  situation 
changes;  to  understand  the  meaning  of  developing  information 
patterns  by  helping  him  to  create  and  evaluate  hypotheses  about  their 
meaning;  and  fmally,  to  help  in  selecting  appropriate  responses. 

On  the  other  hand,  CINCNORAD  emphatically  does  not 
want  an  information  system  which  he  must  fight  to  get  what  he 
needs  when  he  needs  it  -  it  must  be  "user-friendly"  and  extremely 
responsive  to  his  needs.  Nor  does  he  want  an  information  system 
which  tries  to  make  his  assessments  and  decisions  for  him,  either 
implicitly  or  explicitly. 


THE  nW&A  INFORMATION  SYSTEM 

The  ITW&A  information  system  consists  of  all  of  the  sen¬ 
sors  (both  space-  and  ground-based),  communication  links,  intelli¬ 
gence  sources,  world-wide  weather  data  sources,  etc.,  together  with 
the  personnel  and  organization  of  the  CINCNORAD  staff.  The 
CINC  must  utilize  this  system  to  make  time-critical  assessments  and 
decisions  regarding  missile  and  air  attacks  against  both  the  conti¬ 
nental  United  States  and  its  allies. 


Study  sponsored  by  U.S.  Air  Force,  Armstrong  Aerospace  Medical  Research 
Laboratory,  Wright-Paiierson  Air  Force  Base,  Ohio  under  subcontract  to 
Systems  Development  Corporation. 


From  the  perspective  of  control  and  estimation  theory,  there 
appe^  to  be  three  separate  "state  spaces"  involved  in  the  CINCs 
decisionmaking  activities.  As  depicted  in  Fig.  1,  they  are:  (1)  the 
information  space,  which  is  large  and  continually  changing;  (2)  the 
situation  space,  which  is  small  and  represents  all  feasible  correlated 
patterns  (i.e.,  possible  hypotheses  or  meanings)  of  the  data  in  the 
information  space;  and  (3)  the  action  space,  which  represents  the 
feasible  actions  which  the  CINC  can  take  and  therefore  is  highly 
constrained.  The  integration  and  assessment  process  serves  to  map 
the  information  space  onto  the  situation  space,  while  the  decision 
process  serves  to  map  the  situation  space  onto  the  action  space  (in 
each  case,  a  many-to-few  mapping).  These  processes  require  that 
all  of  the  major  centers  in  NORAD  share  key  information  among 
themselves  in  order  to  provide  the  needed  context  for  reducing  both 
assessment  uncertainty  and  response  time. 


INTEGRATION 
AND  ASSESSMENT 


R-4674 


Figure  1.  The  ITW&A  Information  System. 


Assume  for  example  that  CINCNORAD  receives  a  critical 
event  report  including  event  type,  location,  and  time;  and  that  this 
report  could  signify  one  of  two  possible  situations: 

(1)  A  real  attack  is  imminent  or  already  taking  place.  In 
this  case  the  correct  action  is  to  transmit  an  attack 
warning  and  alert  all  strategic  forces. 

(2)  No  attack  is  imminent  or  taking  place  -  the  perceived 
event  is  due  to  accident,  storm,  miscommunication, 
exercise,  foreign  or  domestic  test  launch,  etc.  In  this 
case  the  correct  action  is  simply  to  report  the  situation 
and  insure  that  no  other  commands  may  be  mistaking  it 
for  something  more  serious. 

Given  the  information  input,,  the  two  possible  meanings  are  paral¬ 
leled  by  two  possible  types  of  errors: 
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Figure  2.  Fusion:  The  Data  Integration  and  Assessment  Process. 


(1)  False  alarm,  resulting  in  alerted  forces  even  though  no 
attack  is  taking  place. 

(2)  False  dismissal,  resulting  in  delayed  alert  even  though 
a  real  attack  is  taking  place. 

aearly,  the  cost  of  a  false  dismissal  can  far  overshadow  the  cost  of 
temporally  alerting  forces;  on  the  other  had,  a  continuing  sequence 
of  false  alarms  can  result  in  decreased  decision  sensitivity,  as 
occurred  soon  after  the  DEWLine  was  originally  activated. 

Thus,  discrimination  among  alternative  situation  perceptions 
is  critical,  and  the  ITW&A  system  must  provide  the  capability  for 
improving  this  discrimination,  i.e.,  for  reducing  uncertainty. 


FUSION:  THE  DATA  INTEGRATION  AND  ASSESSMENT 
PROCESS 

As  noted  by  Wohl  (1981),  the  process  of  militpy  decision¬ 
making  is  characterized  by  two  distinct  types  of  uncertainty: 

‘  •  Situation  uncertainty  (”What  is  the  situation?”) 

•  Action  uncertainty  ("What  should  be  done  about  it?”) 

Situation  uncertainty  is  associated  with  the  mapping  of  information 
space  into  situation  space,  while  action  uncertainty  is  associated  wiA 
the  mapping  of  situation  space  into  action  space.  The  focus  in  this 
paper  is  on  situation  uncertainty,  since  one  of  CINCNORAD's  pri¬ 
mary  responsibilities  is  that  of  accurate  and  timely  attack  assess¬ 
ment,  warning,  and  characterization.  The  process  by  which  situa¬ 
tion  uncertainty  is  reduced  has  been  termed  "Fusion"  (sometimes 
"Ops/Intel  Fusion”)  by  the  military,  by  which  is  meant  the  integra¬ 
tion  and  assessment  of  sensor,  intelligence,  and  other  relevant  data 
in  order  to  provide  a  clear  and  complete  situation  picture  with 
uncertainty  minimized  to  the  greatest  possible  extent.  At  NORAD, 
at  the  time  of  this  study,  the  process  was  called  Integration  and 
Assessment. 

Figure  2  summarizes  the  fusion  or  data  Integration  and 
Assessment  (I&A)  process.  It  involves  three  major  activities: 
Monitoring,  Categorization,  and  Assessment.  Sensor  data  is  shown 
as  the  primary  input  to  the  Monitoring  process,  which  involves  the 
blocking  or  filtering  out  of  unwanted  or  irrelevant  data  (e.g.,  the 
moon  or  flocks  of  birds  as  targets),  as  well  as  aggregation  of  data 
from  several  sensors.  Validation  of  the  sensor  data  is  also 
performed  here. 

The  filtered,  aggregated  and  validated  data  then  enters  the 
Categorization  process,  which  involves  correlation  of  multi-source 
data  (e.g.,  track  correlation  from  multiple  radars)  and  interpretation 
(e.g.,  test  launch,  ripple  attack).  Finally,  the  Assessment  process 
involves  the  integration  of  all  available  data;  the  formulation  of  a 
single  succinct  situation  summary;  and  the  final  confirmation  of  that 
summary  prior  to  taking  appropriate  action.  As  indicated  in  Fig.  2, 
each  of  these  three  processes  also  makes  use  of  data  from  past 
history,  independent  sources,  and  trend  data,  as  well  as  from 
intelligence  sources. 

The  over-all  I&A  process  shown  in  Fig,  2  is  what  maps  the 
information  space  onto  the  situation  space  in  Fig.  1.  But  confidence 
in  die  accuracy  of  the  mapping  requires  that  questions  be  asked  to 
test  that  mapping,  which  is  itself  a  hypothesis,  about  the 


situation.  The  kinds  of  questions  which  trained  decisionmakers  ask 
generally  take  the  form  of  hypothesis  tests  to  reduce  situation , 
uncertainty:  i 

If  hypothesis  A  is  correct,  then  we  should  be  observing  the ; 

occurrence  of  certain  other  related  and  predictable  events. 

Are  they  being  observed? 

Experience  with  decisionmakers  and  their  staffs  in  high-level  head¬ 
quarters  indicates  that  they  may  ask  a  series  of  questions  about 
almost  anything,  sometimes  (and  often  to  the  consternation  of  the 
staff)  with  no  apparent  relation  to  the  problem  at  h^d.  However, 
careful  examination  of  the  situation  faced  by  the  decisionmaker  at  a 
given  time  vis-a-vis  the  questions  being  asked  at  that  time  strongly 
suggests  that  they  are  testing  their  existing  situation  perception  and 
examining  alternative  hypotheses.  This  "Integration  and  Assess¬ 
ment”  iJ&A)  Refinement  Loop"  is  shown  in  Fig.  3. 


QUERIES  TO  REFINE 
ASSESSMENT 


INTEGRATION 
AND  ASSESSMENT 


(^^D^ISIONMAK^ 


R-4660 


Figure  3,  The  Integration  and  Assessment  Refinement  Loop. 


A  unique  example  of  how  the  I&A  Refinement  Loop  oper¬ 
ates  is  shown  in  Fig.  4.  Suppose  that  four  alternative  hypotheses 
(A,  B,  C,  and  D)  are  supportable  given  the  data  at  hand  (the  location 
marked  by  an  "x"  represents  ground  truth).  The  size  of  the  circles 
represents  the  degree  of  uncertainty  associated  with  each  hypothesis. 
Note  that  by  reducing  the  uncertainty  associ^ed  with  hypotheses  B, 
C,  and  D,  we  are  left  with  hypothesis  A  by  simple  process  of  elimi¬ 
nation.  Further  reduction  of  uncertainty  regardirig  hypothesis  A  is 
urmecessary.  Many  military  situations  are  of  this  and  can  be 
resolved  by  "asking  the  right  questions." 


Figure  4.  Results  of  Refinement  Process. 
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•  MULTIPLE  EVENTS 

•  HISTORIC  DATA 

•  PROCESSED  a  COMBINED 

•  TRENDS  AND  SUMMARIES 

•  MULTIPLE  SPACES  FROM 
MULTIPLE  SOURCES 


•  DECISION  CRITERIA 

•  "BIG  PICTURES- 

•  SITUATION  ASSESSMENTS 

•  COMBINED  INDICATORS 


•  MAJOR  OPTION  1 

-  ACTION  1A 

-  ACTION  1B 
«  ACTION  1C 

•  MAJOR  OPTION  2 
•  ACTION  2A 

-  ACTION  2B 


-  ACTION  2X 

R-46S7 


Figures.  Current I&A Method:  Diagnostic. 


DIAGNOSTIC  METHOD  OF  INTEGRATION  AND 
ASSESSMENT 

The  I&A  method  in  use  at  the  time  of  this  study  (1985)  is 
depicted  in  Fig,  5.  Because  most  of  the  processing  activity  occurs 
after  detection  and  recognition  of  the  initial  events,  the  term 
"diagnostic"  is  used  to  describe  this  process.  In  other  words,  the 
events  are  considered  as  symptoms,  and  the  purpose  of  I&A  is  to 
reliably  associate  a  most  likely  cause  with  these  symptoms.  The 
generation  and  testing  of  alternative  hypotheses  concerning  possible 
causes  is  the  most  critical  of  the  diagnostic  activities. 

The  relatively  short  time  of  flight  of  enemy  ICBMs  and 
SLBMs  requires  that  the  diagnostic  process  be  performed  under 
high  time-stress  conditions.  Raw  sensor  data  is  first  pre-processed 
at  the  sensor  sites  (or  at  the  Ground  Entry  Points  for  satellite  data)  to 
eliminate  sensor  false  alarms.  This  pre-processed  data  then  enters 
the  NORAD  Space  Defense  Operations  Center  (SPADOC),  Here, 
data  iSrom  multiple  sensors  are  combined  and  correlated  to  increase 
confidence  in  event  detection  and  minimize  the  incidence  of  mis- 
judgments  of  event  type.  This  processed  data  now  becomes  part  of 
the  "Inforrpation  Space"  previously  described.  The  initial  events 
serve  to  trigger  requests  for  relevant  information,  which  in  turn 
spawn  additional  processing  activity  in  SPADOC  to  further  correlate 
or  match  the  event  data  with  information  received  in  oier  NORAD 
operational  centers  (Air  Defense,  Intelligence,  Weather,  etc.).  Thus, 
the  purpose  of  this  processing  activity  is  to  sharpen  the  situation 
perception  and  to  reduce  the  likelihood  of  errors  both  in  identifying 
event  patterns  and  in  discriminating  among  their  possible  interpreta¬ 
tions  as  shown  in  Table  1. 

Table  1.  Possible  Interpretations  of  Sensor  Data _ 

-  Non-threatening  enemy  action 

-  Natural  phenomenon 

-  Unusual  condition  of  own  system 

-  Third-world  or  terrorist  attack 

-  Limited  attack 

-  Saturation  attack 
_ -  etc. 


PROPOSED  PROACTIVE  METHOD  OF  INTEGRATION  AND 
ASSESSMENT 

The  proposed  approach  is  to  perform  most  of  the  diagnostic 
acti\dties  continuously,  before  any  detection  or  event  reports  are  ever 
received.  Using  the  same  historical  data,  independent  source  data, 
trend  data,  and  intelligence  data,  the  various  NORAD  centers  (the 
Intelligence  Center,  the  Air  Defense  Operations  Center,  the  Missile 
Warning  Center,  etc.)  would  be  tasked  to  continuously  look  for 
patterns  and  trends  in  their  individual  information  sets  which  might 
be  associated  with  specific  developing  situations. 

In  the  proposed  proactive  method,  the  received  data  must  be 
considered  to  te  the  symptoms:  the  purpose  of  the  I&A  process  is  to 
reliably  associate  several  possible  causes  with  these  symptoms, 
•before  any  event  occurrence.  The  detection  and  recognition  of  one 
or  more  events  c^  then  serve  as  an  immediate  discriminant  among 
alternative  situation  perceptions  or  hypotheses,  or  as  confirmation  of 
a  highly  likely  situation.  The  many  critical  diagnostic  activities  pre¬ 
viously  performed  on  request  under  high  time-stress  conditions  are 
now  perforrned  continuously  as  a  matter  of  course,  and  prior  to 
event  detection.  Thus,  the  proactive  method  serves  to  greatly 
"stretch"  the  time  available  for  hypothesis  generation  and  testing. 

QOMPARATTVE  EXAMPLE:  SOVIET  TRST  LAI  INGH 

Table  2  has  been  constructed  to  aid  in  exploring  the  differ¬ 
ences  between  the  diagnostic  and  proactive  methods  of  integration 
and  assessment.  Note  the  fundamental  difference  between  the 
sequences  in  Columns  A  and  B.  The  "Diagnostic"  sequence  in  Col¬ 
umn  A  is  triggered  by  the  occurrence  of  the  launch  event  in  step  1, 
resulting  in  tiie  immediate  convening  of  the  "Beige  loop"  conference 
(step  2)  among  key  decision  personnel  in  the  various  Centers  within 
NORAD.  Steps  3  through  7  must  then  intervene  before  the  Soviet 
test  launch  is  confirmed  and  the  ITW&A  product  is  disseminated  in 
steps  8  and  9. 

By  way  of  contrast,  the  "Proactive"  sequence  in  Column  B 
shows  how  continual  hypothesis  generation  activity  (based  on  intel¬ 
ligence  and  otiier  data),  taking  place  prior  to  launch  detection,  can 
result  in  significantly  reduced  time  between  launch  detection  and 
dissemination  of  the  ITW&A  product,  as  in  steps  8  through  1 1. 
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Table  2.  Comparative  Example:  Soviet  Test  Launch 


A.  Diagnostic  Method 

B.  Proactive  Method 

1. 

Launch  event  detected,  recognized  and  validated 

1. 

Intel  sources  notify  re: 

activity  buildup  at  known  Soviet  test  launch  site 
(5  to  10  day  advance  notice  of  test  launch) 

2. 

"Beige  loop"  conference  is  convened 

2. 

Intel  sources  notify  re: 

departure  of  Soviet  tracking  and  telemetry  ship 
from  home  port  (3  day  advance  notice  of  test 
launch) 

3. 

Independent  sources  identify  presence  of  Soviet 
tracking  and  telemetry  ship  at  known  sea  location; 
also  report  Soviet  ocean  closures 

3. 

Independent  sources  notify  re: 

ocean  closures  plus  arrival  of  tracking  and 

telemetry  ship  at  known  sea  location 

4. 

Review  of  past  history  shows  presence  of  same 
ship  at  same  location  during  four  previous  test 
launches  from  this  site 

4, 

Review  of  past  history  indicates  presence  of 
same  ship  at  same  location  whenever  launches 
occur  from  this  test  launch  site 

5. 

Review  of  intel  data  shows  activity  buildup  at 
test  launch  site  during  previous  week 

5. 

Prediction  of  launch  site  weather  for  next 

48  hours  further  narrows  event  time 

6. 

No  supporting  indications  can  be  found  for 
alternative  hypotheses 

6. 

Alert  sensor  stations  to  probable  test  launch 
within  24  hours 

7. 

Event  interpreted  as  Soviet  test  launch,  based  on 
ocean  closures;  launch  location,  time,  and  type; 
and  trajectory 

7. 

No  supporting  indications  found  for  alternative 
hypotheses 

8. 

Test  launch  is  confirmed 

8. 

Launch  event  detected,  recognized  and 
validated 

9. 

Appropriate  ITW&A  product  is  disseminated 

9. 

Conference  is  convened 

10. 

Test  launch  is  confirmed 

11. 

Apppropriate  ITW&A  product  is  disseminated 

CONCLUSIONS 

It  is  clear  that  post-event  diagnosis  is  done  in  "spurts."  It  is 
based  on  mutual  requests  for  information,  which  must  take  place 
under  high-stress  conditions.  This  takes  time,  and  in  addition, 
requires  multiple  information  requests  and  coordination  among  the 
information  centers  in  NORAD. 

The  proposed  proactive  method  is  not  without  cost,  how¬ 
ever.  While  it  saves  critical  time,  it  requires  additional  and  continu¬ 
ous  processing  effort.  Even  though  most  of  this  effort  can  take 
place  well  before  launch  detection,  it  clearly  would  require  addition^ 
resources  as  well  as  changes  in  the  way  in  which  intelligence  data  is 
handled  and  communicated  among  the  NORAD  centers.  Also,  the 
patterned  movement  of  the  Soviet  tracking  ship  might  well  be  a 
"cover"  for  a  hostile  AS  AT  launch,  so  that  the  additional  processing 
effort  must  include  a  specific  hypothesis  to  be  tested  for  just  such  an 
instance. 

It  is  interesting  to  note  that  much  of  the  required  processing 
for  the  proactive  approach  was  already  being  done  at  NORAD  at  the 
time  of  this  study,  but  the  information  was  not  adequately  organized 
within  the  NORAD  centers,  nor  was  it  being  appropriately  shared 
among  the  centers.  Finally,  it  must  be  remembered  that  the  most 
critical  processing  was  being  carried  out  under  post-event,  high- 
stress  conditions. 


In  any  case,  it  is  also  clear  that  the  various  NORAD  center 
must  learn  to  anticipate  each  others  information  needs  as  strategic 
threat  situations  develop,  before  any  launch  events  occur  and  the 
CINC  gets  involved. 

One  final  point  is  worth  making  with  regard  to  the  implica¬ 
tions  of  the  diagnostic  vs.  proactive  methods  for  the  Strategic 
Defense  Initiative  (SDI)  program.  The  Weapons  Enable  decision 
will  perhaps  be  the  most  critical  single  command  and  control  deci¬ 
sion  for  which  humans  must  take  responsibility  in  an  operational 
Space  Defense  system.  The  various  SDI  architecture  studies  have 
repeatedly  demonstrated  the  severe  impact  of  decision  delays  on 
system  effectiveness  (i.e.,  on  reent^  vehicle  leakage  rates).  For 
this  reason,  proactive  data  fusion  will  more  than  likely  be  an  abso¬ 
lute  necessity  for  an  effective  Space  Defense  system. 
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A  template  system,  which  incorporates  time-event  models  of  hostile  activities,  helps  analysts 
infer  hostile  objectives  and  plans  from  a  pattern  of  observables.  This  system  represents  and 
manages  uncertainties  about  the  identity  and  time  of  events,  the  relationships  among  events,  and 
the  event  participants.  Uncertainties  are  represented  as  different  types  of  intervals.  Inference 
and  evidence  combination  are  accomplished  through  operations  on  these  intervals. 

The  uncertain  time  of  an  event  is  represented  by  two  intervals:  a  broader  interval 
representing  the  full  range  of  possible  times  of  the  event,  and  a  narrower  interval  representing 
the  typical  times  for  the  event.  The  former  interval  is  highly  reliable  but  often  very  large.  The 
latter  interval,  which  is  based  on  expert  judgment  of  what  usually  occurs,  is  often  much  more 
precise  but  much  less  dependable. 

Times  for  events  are  estimated  from  the  times  of  reported  activities  and  template  models  of 
hostile  operations.  Each  template  model  describes  the  events  in  an  operation,  the  range  of  typical 
and  possible  durations  for  each  of  these  events,  and  the  ranges  of  typical  and  possible  time 
intervals  between  events. 

There  are  two  steps  in  computing  the  estimated  time  of  a  future  event  from  the  estimated  time 
of  an  earlier  event.  In  the  first  step,  the  time  of  the  future  event  is  projected  from  the  earlier 
one  using  the  duration  and  time  interval  information  in  the  template.  In  the  second  step  this 
projected  interval  is  combined  with  the  prior  estimate  of  the  future  event  to  produce  a  re  ined 
estimate . 

The  computational  methods  for  estimating  the  "possible"  and  "typical"  times  differ  in  both  of 
these  steps.  The  estimate  of  possible  times  uses  bounding  techniques  while  the  estimate  of  typical 
times  approximates  probability  methods. 

In  projecting  the  earliest  possible  time  of  a  future  event,  the  earliest  start  time  of  the 
first  event  is  added  to  the  shortest  time  interval  between  the  earlier  and  later  event.  The  latest 
start  time  is  computed  similarly.  When  refining  this  estimate  by  combining  the  estimate  from  two 
sources,  the  estimates  from  each  source  are  treated  as  completely  reliable.  The  earliest  possible 
start  time  is  the  later  of  the  two  estimates  of  earliest  start  time  in  each  source,  and  the  latest 
possible  start  time  is  the  earlier  of  these  estimates. 

In  projecting  the  earliest  and  latest  typical  start  times  of  a  future  event,  the  earliest  and 
latest  start  times  of  the  earlier  event  and  the  earliest  and  latest  likely  time  intervals  between 
events  are  considered  to  reflect  the  mean  and  standard  deviations  of  two  independent  normal 
distributions.  The  earliest  and  latest  start  time  of  the  future  event  reflect  the  sum  of  these 
distributions.  When  refining  this  estimate  by  combining  evidence  from  two  sources,  each  source  is 
considered  to  provide  independent  estimates  of  typical  times,  based  on  different  data  and 
considerations.  An  event  time  in  the  refined  estimate  is  considered  to  be  within  the  typical 
interval  if  at  least  one  source  of  evidence  considers  the  time  to  be  typical,  and  the  other 
considers  it  to  be  typical  or  close  to  typical. 
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different  types  of  evidence  about  the  times 
of  events  are  fused. 


INTRODUCTION 

Template-based  situation  assessment 
infers  possible  ongoing  hostile  operations 
from  a  pattern  of  reported  observables .  Each 
template  specifies  the  observations  expected 
given  that  a  particular  hostile  operation  is 
being  conducted.  An  operation  associated 
with  a  template  is  inferred  when  enough  of 
the  observations  specified  by  the  template 
are  reported. 

One  of  the  major  difficulties  in 
developing  workable  template-based  systems  is 
defining  templates  that  can  accommodate 
operational  variability.  While  it  may  be 
easy  to  develop  a  template  that  describes  one 
particular  "typical"  hostile  operation,  it  is 
much  more  difficult  to  develop  more  flexible 
templates  able  to  describe  a  range  of 
possible  operations.  Such  flexibility  is 
required  in  a  templating  system,  for  very  few 
hostile  operations  are  likely  to  occur  in  an 
exactly  "typical"  fashion.  A  template  unable 
to  recognize  these  less  typical  operations  is 
unlikely  to  be  very  useful. 

The  approach  used  here  for  accommodating 
operational  variability  is  motivated  by 
models  of  human  memory  organization  and 
information  processing  (Ref  1) .  This  model 
has  led  to  templates  that  represent  data  in 
the  kind  of  "flexible  way  that  reflect  human 
tolerance  for  vagueness,  imprecision,  and 
quasi-inconsistencies"  (Ref.  2). 

This  flexibility  and  tolerance  for 
imprecision  arises  from  the  way  that 
uncertainty  is  represented  within  the 
templating  system.  There  are  many  different 
types  of  uncertainty  in  a  template: 
uncertainty  about  the  type  of  operational 
events,  about  the  times  of  these  events,  and 
about  the  types,  numbers,  missions,  and 
locations  of  hostile  platforms.  Each  of 
these  must  be  represented  and  managed. 

This  paper  describes  how  one  of  these 
types  of  uncertainty,  temporal  uncertainty, 
is  handled.  It  describes  how  different  types 
of  temporal  uncertainty  are  represented,  how 
uncertain  times  are  projected,  and  how 


The  following  example  introduces  the 
principal  concepts  in  managing  uncertainty  in 
the  templating  system.  It  shows  the  kinds  of 
inferences  about  the  times  of  events  that  are 
made  by  the  templating  system.  The  method 
for  computing  these  times  in  described  in  the 
sections  following  this  example. 


AN  OPERATIONAL  EXAMPLE 

In  this  example,  a  U.S.  force  in  the 
Indian  Ocean  is  tasked  to  evacuate  U.  S. 
citizens  from  Pakistan.  Intelligence 
indicates  a  possible  Soviet  air  strike 
against  the  carrier,  A  Soviet  reconnaissance 
flight  is  reported  to  be  in  progress.  Its 
take  off  time  is  not  certain,  but  was 
probably  between  0830  and  0840,  and  was 
certainly  not  before  0800  or  after  0900.  We 
have  been  asked  to  appraise  the  commander  of 
the  progress  of  this  possible  strike,  and  to 
advise  him  as  early  as  possible  when  the 
launch  may  occur. 

Figure  1  illustrates  our  estimate  of  the 
times  of  some  of  the  key  events.  These 
events  are  the  take  off  of  the  reconnaissance 
aircraft,  the  time  that  this  aircraft  will 
likely  find  us,  the  time  that  the  strike 
aircraft  with  arrive  in  the  area  patrolled  by 
our  combat  air  patrol  (CAP) ,  and  the  time 
that  the  launch  will  occur. 

None  of  these  times  can  be  estimated 
exactly,  but  each  can  be  estimated 
approximately.  Figure  1  represents  these 
estimated  times  as  two  time  intervals.  The 
outer  interval,  colored  gray,  represents  the 
range  of  possible  times.  The  inner  interval, 
colored  black,  represents  the  most  likely 
times.  The  intervals  for  "take  off"  reflects 
our  recently  received  report. 

This  hybrid  representation  of  possible 
and  likely  times  is  intended  to  combine  the 
advantages  of  estimates  based  on  capability 
with  the  advantages  of  estimates  based  on 
hostile  doctrine  and  tactics.  The  estimates 
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UPDATED  ESTIMATE  OF  LAUNCH  TIME 
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Figure  1.  Estimated  times  of  hostile  events 
based  on  reported  take  off. 


of  possible  times,  which  are  based  on  hard 
intelligence  data  about  hostile  forces,  are 
highly  reliable.  In  Figure  1  the  bounds  on 
possible  times  are  based  on  fuel  capacity, 
nearest  and  farthest  possible  launch  points, 
and  maximum  aircraft  speed.  As  suggested  by 
the  figure,  these  estimates  tend  not  to  be 
very  constraining,  permitting  a  range  which 
may  be  very  large. 

The  estimates  of  likely  times  are  based 
on  typical  tactics,  likely  aircraft  launch 
locations,  typical  types  of  aircraft,  and 
average  luck  in  finding  the  U.S.  forces.  As 
suggested  in  Figure  l,the  bounds  on  typical 
times  can  be  much  narrower  than  the  bounds  on 
the  possible  times.  Unfortunately,  there  is 
no  guarantee  that  the  actual  time  of  the 
event  will  fall  within  these  bounds,  for 
hostile  forces  are  free  to  conduct  an 
operation  in  an  atypical  way. 

These  estimates  illustrate  inferences 
that  may  be  made  from  a  single  contact  report 
of  the  probable  and  possible  take  off  times 
for  a  reconnaissance  aircraft,  and  from 
knowledge  about  hostile  capabilities, 
doctrine,  and  tactics. 

Initially,  the  estimates  made  from  such 
reports  may  be  very  uncertain.  As  additional 
reports  are  received,  however,  they  can  often 
be  considerably  refined.  In  our  example.  We 
assume  that  the  following  reports  will  be 
received: 

Reported  Activity _ Time,.  a£..  E-Ygat 


OUR  FORCE 
LOCATED 


STRIKE  FORCE 
DEPLOYED 


STRIKE  FORCE 
IN  AREA 


MISSILE 

LAUNCHED 


1200  1400  1600  1800 


TYPICAL  TIMES  POSSIBLE  TIMES 


Figure  2.  Estimated  launch  time  after 
receiving  each  report . 


physically  possible,  however,  for  the  launch 
to  occur  much  earlier  or  much  later.  The 
latest  time  assumes  that  the  launch  will  be 
delayed  until  the  last  possible  moment, 
limited  only  by  the  fuel  capacity  of  the ^ 
targeting  and  strike  aircraft.  The  earliest 
time  assumes  that  the  reconnaissance  aircraft 
is  very  lucky  finding  us,  that  the  fastest 
strike  aircraft  will  be  used  and  will  take 
off  immediately. 

Each  of  the  reports  about  the  hostile 
forces  conveys  information  about  the  progress 
of  the  attack,  and  each  contains  information 
enabling  the  estimated  launch  time  to  be 
refined.  Note  that  in  all  cases,  until  the 
launch  itself  is  detected,  the  range  of 
possible  times  is  significantly  broader  than 
the  range  of  typical  times. 


TRMPT.ATE  REPRESENTATION  OF  EXPERT  KNOWLEDGE 

In  the  example  the  estimates  of  event 
times  were  based  on  reports  of  hostile 
activity  and  an  understanding  of  hostile 
force  capabilities,  doctrine,  and  tactics, 
and  were  computed  using  mathematical  methods 
for  managing  uncertain  times.  The 
understanding  about  hostile  forced  required 
for  these  calculations  is  contained  within 
reference  templates  described  immediately 
below.  The  mathematics  is  described  in  the 
following  section. 


Reconnaissance 

Possible : 

0800 

to 

0900 

take  off 

Probable : 

0830 

to 

1040 

Our  force 

Possible : 

1040 

to 

1140 
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1120 

to 

1130 

Strike  force 

Possible : 

1140 

to 

1210 

deployed 

Probable : 

1140 

to 

1200 

Strike  force  At  1450 

in  area 


Missile  launched  At  1522 

Figure  2  illustrates  how  the  estimated 
time  of  the  hostile  missile  launch  can  be 
continuously  refined  after  each  of  these 
reports.  After  the  initial  reconnaissance 
report,  it  is  estimated  that  the  launch  will 
likely  occur  between  1800  and  1900.  It  is 


A  reference  template  is  a  time-event 
model  that  describes  a  type  of  military 
operation.  Each  reference  template  models  an 
entire  class  of  operations,  rather  than  just 
a  single  specific  one.  Each  describes  not 
only  the  typical  activities  that  comprise  the 
operation,  but  also  the  likely  and 
permissible  variations  in  these  activities. 
Templates  used  for  military  situation 
assessment  also  specify  the  indicators  and 
observables  associated  with  the  operation. 
Reference  templates  are  developed  by 
operational  experts,  and  capture  expert 
knowledge  about  hostile  tactics  and  doctrine. 

Figure  3  summarizes  the  reference 
templates  used  in  the  preceding  example.  The 
"Find  and  Strike"  template  describes  the 
overall  attack.  The  four  small  templates  at 
the  bottom  of  the  figure  describe  the  main 
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RECONNAISSANCE 


LOCALIZE 


TAKEOFF 


FIND  SURVEY  REPORT  TASKING  ARRIVE  JAM  LAUNCH 

TARGET  FORCES  AREA 


DOCTRINAL  KNOVYLEDGE: 


DEFINED  TIME  INTERVALS 


TYPES  OF  EVENTS  DURATIONS  OF  EVENTS 

OBSERVABLES  IN  EVENTS  TIME  INTERVALS  BETWEEN  EVENTS 


Figure  3.  Template  represention  of  hostile 

capabilities,  doctrine,  and  tactics. 


events  in  the  attack.  In  each  of  the 
templates  the  rows  denote  the  major 
participants,  and  time  runs  along  the 
horizontal  axis.  Important  events  are 
depicted  on  the  row  of  the  participant 
performing  them,  placed  along  the  horizontal 
axis  at  the  time  that  they  typically  occur. 

The  "Find  and  Strike"  template  has  two 
rows.  The  top  row  depicts  the  activities  of 
the  reconnaissance  platform.  It  is 
responsible  for  two  events,  reconnaissance, 
and  localization.  In  reconnaissance  the 
aircraft  finds  the  U.  S.  forces.  During 
localization,  it  tracks  the  priority  targets. 
The  second  row  depicts  the  activities  of  the 
strike  force.  Deployment  is  transit  to  the 
target  area.  The  strike  entails  penetration 
of  our  defenses  and  launch  and  guidance  of 
the  missile. 

There  are  two  different  types  of  times 
specified  in  reference  templates:  times 
between  events  and  event  durations. 

The  arrows  in  Figure  3  indicate  those 
event  pairs  that  have  explicitly  defined 
temporal  relationships  with  each  other. 

These  relationships  may  specify  the  time 
interval  between  the  beginnings  of  two 
events,  between  the  ends  of  two  events,  or 
between  the  beginning  of  one  event  and  the 
end  of  another.  In  Figure  3  there  is  a  time 
relationship  between  the  end  of 
"Reconnaissance"  and  the  beginning  of 
"Localize".  This  relationship  indicates  that 
localization  cannot  begin  until  the  U.  S. 
forces  have  been  found  and  reconnaissance  is 
over.  There  is  also  a  defined  relationship 
between  the  beginning  of  reconnaissance  and 
the  end  of  localization.  This  relationship 
limits  the  total  time  of  these  two  events  to 
the  fuel  capacity  of  the  reconnaissance 
aircraft . 

Each  of  these  time*  intervals  may  be 
uncertain.  Like  the  uncertain  times  for  the 
hypothesized  events  of  Figure  1,  the  time 
intervals  are  also  represented  by  two 
intervals :  an  interval  showing  the  bounds  on 

what  is  possible,  and  an  interval  showing  the 
bounds  on  what  is  most  likely  or  typical. 
These  two  intervals  are  represented  by  four 
numbers.  For  example  the  temporal 
relationship  between  the  end  of  event  "A" 
(Reconnaissance)  and  the  beginning  of  event 


"'b"  (Localize)  is  represented  by  the  four 
tuple 

Time  Interval  AB  =  (ABi,  AB2,  AB3,  AB4) 

where 


ABi  =  shortest  possible  time  interval 
AB2  =  shortest  likely  time  interval 
AB3  =  longest  likely  time  interval 
AB4  =  longest  possible  time  interval 

The  duration  of  events  may  also  be 
uncertain.  Each  duration  is  represented  by 
four  numbers  denoting  the  possible  and 
typical  time  intervals  between  the  beginning 
and  end  of  the  event  .  These  intervals  are 
defined  in  the  detail  templates  for  the 
events.  In  Figure  3,  for  example,  the 
duration  of  the  event  "Reconnaissance"  in  the 
"Find  and  Strike"  template  is  defined  as  the 
uncertain  time  interval  between  the  events 
"Take  off”  and  "Find  Target"  in  the  template 
"Reconnaissance"  shown  at  the  bottom  of  the 
figure . 

The  hybrid  representation  of  uncertainty 
for  times  between  events  and  for  event 
durations  has  the  same  advantage  as  the 
hybrid  representation  for  the  estimated  times 
of  ongoing  events.  The  outer  range,  the 
range  of  possible  times,  is  highly  reliable 
but  may  often  be  extremely  broad.  The  inner 
times,  the  range  of  likely  times,  is 
considerably  less  reliable,  but  is  often  much 
more  precise. 

In  illustrations  of  templates,  event 
duration  is  suggested  by  the  length  of  the 
boxes  representing  each  event. 


CALCULATING  THE  UNCERTAIN  .TIME_QF_EYENTS 

Uncertain  event  times,  as  shown  in 
Figure  1,  are  represented  by  four  numbers. 

For  example,  the  time  that  event  "A"  (Take 
off)  begins  is  represented  by  the  numbers 

Time  A  =  (Ai,  A2,  A3,  A4)  where 

Ai  =  earliest  possible  start  time 

A2  =  earliest  likely  start  time 

A3  =  latest  likely  start  time 

A4  -  latest  possible  start  time 

The  uncertainty  ranges  in  the  times  of  a 
future  event  are  computed  from  the 
uncertainty  ranges  of  an  earlier  event  in  two 
steps.  In  the  first  step,  an  estimate  of  the 
tine  of  the  second  event  is  computed  by 
adding  the  uncertain  time  interval  between 
these  events  to  the  uncertain  time  of  the 
first  event.  In  the  second  step,  this 
projected  time  is  combined  with  the  prior 
estimate  of  the  time  of  the  second  event  to 
produce  a  refined  estimate  of  the  event's 
time.  These  steps  are  central  to  data 
fusion,  for  they  combine  the  evidence  about 
the  time  of  the  earlier  event  with  the 
evidence  about  the  time  of  the  second  event 
to  produce  a  better  estimate  of  the  second 
event  time. 

Step  1;  Projecting  Future  Time..s. 

The  bounds  for  possible  and  typical 
projected  times  are  calculated  using  separate 
formulae . 


no 


Formula  for  the  bounds  on  possible --t.im£.s. 

BlfromA  =  Ai  +  ABi  Earliest  possible 
time  for  B 

B4fromA  =  A4  +  AB4  Latest  possible  time 
for  B 

Formula  for  the  bounds  on  typical  times 

Let  0  =  .5  *(A2  +  A3  +  AB2  +  AB3) 

Let  S  =  .25  *  (A2  -  A3) 2  +  {AB2  -  AB3)2 

B2fromA  =  U  -  S  Earliest  likely  time 
for  B 

B3fromA  =  U  +  S  Latest  likely  time 

for  B 

The  bounds  on  the  possible  intervals, 
BlfromA  and  B4fromA  are  outer  estimates  of 
theoretically  possible  times.  The  earliest 
possible  time  of  B,  as  estimated  only  from  A 
and  the  relationship  between  A  and  B,  is  the 
sum  of  the  earliest  possible  time  of  A  and 
the  shortest  possible  time  interval  between  A 
and  B.  Similarly,  the  latest  possible  time 
of  B  is  the  sum  of  the  latest  possible  time 
of  A  and  the  longest  time  interval  between  A 
and  B. 

The  bounds  on  the  typical  intervals, 
B2fromA  and  B3fromAf  are  calculated 
differently.  Because  the  concept  of 
"typical”  resembles  the  concept  of 
"probable",  the  formula  for  these  bounds  are 
based  on  probability  theory,  and  the 
projected  times  behave  as  expected  from 
operations  on  probabilities.  For  instance,  a 
typical  interval  which  is  the  sum  of  two 
other  typical  intervals  is  somewhat  larger 
than  either  of  the  two  contributing 
intervals . 

The  formula  for  the  typical  interval 
treats  the  inner  bounds  of  the  uncertain 
times,  A2  and  A3,  and  the  inner  bounds  of 
uncertain  interval  between  events,  AB2  and 
AB3  ,as  the  standard  deviation  departures 
from  the  mean  of  normal  distributions.  The 
inner  bounds  of  the  projected  time,  B2fromA 
and  B3fromA  /  is  interpreted  as  the  standard 
deviation  departure  from  the  mean  of  the  sum 
of  the  two  normal  distributions. 

Data  Fusion:  Combining  Estimates  Qf  an 
Event *s  Time 

The  refined  estimate  of  a  time  for  an 
event  "B"  is  computed  by  combining  the 
estimates  of  the  event  time  projected  from 
earlier  events  with  the  prior  estimate  of 
this  event  time.  This  process  is  summarized 
in  Figure  4 . 

The  new  estimate  for  time  Bi,  the 
earliest  time  that  B  can  begin,  is  the 
minimum  of  the  times  BlfromA  Bipriorf 
where  Biprior  is  the  prior  estimate  of  the 
minimum  start  time  of  event  B.  This  rule 
reflects  the  premise,  true  within  each 
hypothesis  though  not  necessarily  true 
outside'  the  hypothesis,  that  both  estimates, 
BlfromA  and  Biprior^  are  completely  reliable. 
Consequently,  BifroraA  and  Biprior  both 
provide  absolute  lower  bounds .  The  new  lower 
bound  is  the  greater  of  these  two  numbers. 


Figure  4.  Combination  of  evidence,  refining  an 
estimate  of  event  time. 


Similarly,  the  estimate  of  the  greatest 
possible  time,  B4,  is  the  minimum  of  the  two 
estimates,  BlfromA  and  Biprior- 

The  range  of  typical  values,  B2  to  B3, 
is  set  to  those  values  classified  as  typical 
or  nearly  typical  in  both  contributing 
estimates.  A  particular  time  for  B  will  be 
regarded  as  typical  if  it  is  regarded  as 
typical  in  both  Bprior  and  BfromA-  A  time 
can  also  be  considered  to  be  typical  for 
event  B  if  it  is  regarded  as  typical  in 
either  Bprior  or  B2fromA  and  is  not  regarded 
as  atypical  in  either  estimate.  "Not 
atypical"  is  a  parameter  of  the  program, 
corresponding  in  Figure  4  to  a  height  on  the 
descending  slope  of  the  uncertainty 
representation . 


SUMMARY 

The  estimated  times  of  events  in  a 
situation  assessment  are  often  uncertain. 
These  uncertainties  may  be  represented  by  two 
intervals:  a  broader  interval  representing 

the  full  range  of  possible  times  of  the 
event,  and  a  narrower  interval  representing 
the  typical  times  for  the  event.  The  former 
interval  is  highly  reliable  but  often  very 
large.  The  latter  interval,  which  is  based 
on  expert  judgment  of  what  usually  occurs,  is 
often  much  more  precise  but  much  less 
dependable . 

Times  for  events  are  estimated  from  the 
times  of  reported  activities  and  a  template 
model  of  hostile  operations.  The  template 
model  describes  the  events  in  an  operation, 
the  ranges  of  typical  and  possible  durations 
for  these  events,  and  the  range  of  typical 
and  possible  time  intervals  between  events. 

There  are  two  steps  in  computing  the 
estimated  time  of  a  future  event  from  the 
estimated  time  of  an  earlier  event.  In  the 
first  step,  the  time  of  the  future  event  is 
projected  from  the  earlier  one  using  the 
duration  and  time  interval  information  in  the 
template.  In  the  second  step  this  projected 
interval  is  combined  with  the  prior  estimate 
of  the  future  event  to  produce  a  refined 
estimate . 

The  computational  methods  for  estimating 
the  "possible"  and  "typical"  times  differ. 

The  estimate  of  possible  times  uses  bounding 


in 


techniques  while  the  estimate  of  typical 
times  approximates  probability  methods. 

These  methods  of  representing  and 
managing  temporal  uncertainty  are  embedded 
within  a  system  for  inferring  possible 
hostile  actions  from  a  pattern  of  observed 
activities.  In  the  near  future  the  system 
will  represent  and  manage  uncertainties  about 
platforms  identities  and  locations.  The 
system  will  represent  these  uncertainties  in 
the  same  way  way  that  time  uncertainty  is 
represented,  as  a  range  of  typical  values  and 
a  range  of  possible  values. 
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ABSTRACT 

A  constraint-based  approach  to  uniformly  combining  information  from  multiple  representations  and  sources 
of  sensory  data  is  described.  The  approach  is  important  to  research  in  intermediate  grouping,  knowledge- 
based  model  matching,  and  information  fusion.  The  techniques  presented  extend  the  capabilities  of  an 
earlier  system  that  applied  constraints  to  attributes  of  single  types  of  extracted  image  events  called  tokens. 
Relational  measures  are  defined  between  symbolic  tokens  so  that  sets  of  tokens  across  representations  can 
be  selected  and  grouped  on  the  basis  of  constraint  functions  applied  to  these  relational  measures. 

Since  typical  low-level  representations  involve  hundreds  or  thousands  of  tokens  in  each  representation, 
even  binary  relational  measures  can  involve  very  large  numbers  of  token  pairs.  Control  strategies  for 
ordering  and  filtering  tokens,  based  upon  constraints  on  token  attributes  and  token  relationships,  can  be 
formed  to  reduce  the  computation  involved  in  producing  token  aggregations.  The  system  is  demonstrated 
using  region  and  line  data  and  an  associated  set  of  relational  measures.  The  approach  can  be  naturally 
extended  to  include  tokens  extracted  from  motion,  stereo,  and  range  data. 
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Projects  Agency  (DOD),  ARPA  Order  No.  N00014-82-K-0464,  Air  Force 
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tional  Science  Foundation  under  contract  DCR-8500332. 


113 


Sensor  and  Information  Fusion  From  Knowledge-Based  Constraints 

Allen  R.  Hanson,  Edward  M.  Riseman 
Computer  ^  Information  Science  Department 
University  of  Massachusetts^ 

Amherst,  MA  01003 

Thomas  D.  Williams 
Amerinex  Artificial  Intelligence,  Inc. 

274  N.  Pleasant  Street 
Amherst,  MA  01002 


1  Introduction 


A  major  problem  confronting  vision  systems  whicii  use  mul¬ 
tiple  sensors,  or  which  generate  multiple  low-level  descriptions 
from  image  data,  is  the  coherent  and  consistent  integration  of  in¬ 
formation  contained  in  the  multiple  representations.  Most  vision 
systems  utilize  only  one  type  of  sensory  data  (e.g.,  visible  light, 
SAR,  IR,  range)  and  only  one  type  of  low-level  process  producing 
a  single  type  of  extracted  image  event,  e.g.,  regions  of  a  region 
segmentation.  However,  after  many  years  of  computer  vision  re¬ 
search  [HAN78a,b,HAN87,RIS87|  it  is  clear  that  such  systems 
are  fundamentally  limited  by  their  restricted  and  unreliable  view 
of  the  image  data,  and  consequently  their  performance  must  suf¬ 
fer  by  the  degree  to  which  the  image  descriptions  fail  to  support 
the  system’s  goals. 

More  recently,  multiple  sensors  have  become  more  readily 
available  and  many  algorithms  have  been  developed  for  process¬ 
ing  each  type  of  sensor  data.  For  example,  depth  maps  can  ob¬ 
tained  directly  from  laser  range  data,  and  indirectly  from  motion 
and  stereo  algorithms  that  are  applied  to  pairs  and  sequences  of 
images,  respectively.  It  has  also  become  evident  that  each  low- 
level  process  extracts  only  partial  descriptions  of  the  underlying 
image  structure,  and  that  there  is  a  great  deal  of  redundancy 
which  can  be  profitably  exploited  across  these  descriptions.  Con¬ 
sequently,  the  need  is  becoming  more  acute  for  computer  vision 
systems  to  fuse  the  information  extracted  by  different  types  of 
low-level  vision  algorithms  into  more  coherent  descriptions.  Max¬ 
imum  reliability  can  only  be  achieved  through  processes  that  can 
integrate  information  represented  in  widely  varying  forms. 

To  be  somewhat  more  specific,  consider  the  interpretation  of 
a  road  scene.  The  formation  of  a  ‘road’  hypothesis  should  not 
be  based  on  any  single  type  of  extracted  image  event  (e.g.  re¬ 
gions),  but  rather  on  an  aggregation  of  multiple  types  of  events 
(e.g,  lines,  regions,  and  surfaces)  that  have  specific  relationships 
to  each  other  and  which  contribute  to  the  support  of  the  road 
structure.  For  example,  one  might  like  to  find  a  homogeneous 
region  of  an  expected  intensity  and  color,  bounded  by  two  con¬ 
verging  straight  lines,  and  approximately  covered  by  a  horizontal 
planar  surface. 

Of  course  the  reader  should  not  be  misled  into  an  oversim¬ 
plified  view  of  the  problem;  there  are  extremely  difficult  low- 
level  issues  to  be  dealt  with,  such  as  the  instability  of  segmen¬ 
tation  algorithms  that  leads  to  unpredictable  fragmentation  of 
lines,  regions,  and  surfaces  [BEV87,KOH83,NAG82],  and  incon¬ 
sistencies  between  the  elements  extracted  in  these  representa¬ 
tions.  These  are  problems  that  are  implicit  in  the  nature  (if  the 
problem  of  integrating  unreliable  information  and  will  be  true 


of  all  approaches,  not  just  the  one  presented  here.  Our  view  is 
that  to  fully  integrate  multiple  representations  there  will  need  to 
be  complex  grouping  strategies  that  utilize  the  techniques  pre¬ 
sented  here  as  part  of  a  knowledge-directed  interpretation  process 
[BOL87,DRA87a,b,HAN87,REY87b]. 

In  this  paper,  we  take  the  view  that  information  fusion  can 
be  accomplished  during  later  stages  of  the  interpretation  process, 
rather  than  when  the  image  events  are  first  extracted  (e.g.  by 
attempting  to  directly  integrate  region  and  line  algorithms).  We 
also  believe  this  will  avoid  some  of  the  severe  ambiguity  problems 
encountered  when  performing  interpretation  on  the  independent 
representations  prior  to  information  fusion.  Our  approach  to 
fusion  will  be  illustrated  here  by  extending  a  constraint-based 
object  hypothesis  system  {RIS87|  to  operate  over  multiple  token 
types,  in  this  case  regions  and  lines  extracted  from  the  image 
data.  The  construction  of  the  region  and  edge  representation 
makes  use  of  two  low-level  algorithms:  a  local  histogram-based 
region  segmentation  algorithm  [BEV87,KOH83,NAG82]  and  a 
straight  line  extraction  algorithm  [BUR86].  Examples  of  these 
two  processes  are  shown  in  Figure  5b, c.  When  surface  elements 
extracted  from  depth  maps  are  available,  they  can  be  aggregated 
with  regions  and  lines.  The  techniques  could  also  be  easily  ex¬ 
tended  to  include  fusion  of  information  from  textured  areas,  cor¬ 
ners,  volumes,  and  generally  any  other  token  abstracted  from  the 
same  or  other  sensory  sources. 

There  have  been  a  few  attempts  to  integrate  results  from  mul¬ 
tiple  low-level  processes  operating  on  one  or  more  sensory  sources 
[HAN78b,KOH83,NAS83].  In  the  past,  efforts  to  combine  multi¬ 
ple  processes  operating  on  visual  data  have  typically  involved  the 
integration  of  line  and  region  data,  which  are  the  two  most  com¬ 
mon  types  of  low-level  algorithms  employed.  More  recently,  there 
has  been  an  increasing  number  of  efforts  to  combine  range  and  vi¬ 
sual  data  [ARK87,BES85,SHA86l.  Shafer  and  Thorpe  [SHASe] 
developed  a  blackboard  system  for  the  CMU  NAVLAB  mobile 
vehicle;  in  this  system,  range  data  and  visual  data  are  indepen¬ 
dently  processed  and  combined  during  the  interpretation  pro¬ 
cess.  On  the  other  hand,  Nandhakumar  and  Agarwal  [NAN87] 
combined  the  processing  of  infrared  and  visible  light  images  with 
computational  models  of  the  image  generation  process  to  improve 
the  results  beyond  that  achievable  by  either  process  alone. 
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2  Background 

2.1  The  Intermediate  Symbolic  Representation  (ISR) 

The  most  general  mode!  of  image  interpretation  involves  the  con¬ 
struction  of  a  symbolic  three-dimensional  description  of  a  scene 
from  an  image  or  set  of  images;  a  related  goal  is  the  identifica¬ 
tion  of  a  specific  ‘target’  object  from  background  clutter.  It  is 
generally  accepted  that  a  computer  vision  system  must  perform 
a  variety  of  transformations  of  the  data  during  this  interpreta¬ 
tion  process.  Consequently,  at  a  coarse  conceptual  level,  the  VI¬ 
SIONS  image  understanding  system  is  organized  into  three  levels 
of  processing;  low,  intermediate,  and  high  as  shown  in  Figure 
1.  Currently,  the  low-level,  or  segmentation,  processes  output  a 
symbolic  representation  of  the  data  in  the  form  of  regions  and 
lines.  Attributes,  such  as  color,  texture,  location,  size,  shape,  and 
orientation,  are  then  calculated  for  each  region  or  line.  Interpre¬ 
tation  processes  use  knowledge  of  the  objects  in  the  domain  to 
control  a  set  of  intermediate-level  processes  for  generating  initial 
object  hypotheses  and  reorganizing  the  low-level  data.  World 
(domain)  knowledge  is  then  responsible  for  resolving  these  hy¬ 
potheses  data  into  a  consistent  model  of  the  scene. 

One  of  the  key  abstractions  is  the  transformation  of  pixels,  or 
more  generally  arrays  of  sensory  data,  into  image  events  which 
can  be  named  and  accessed  by  their  properties.  We  refer  to 
the  symbolic  representation  of  an  extracted  image  event,  such  as 
a  particular  region,  line,  curve,  rectangle,  or  surface,  etc.,  as  a 
token]  attributes  are  associated  with  tokens,  and  tokens  partic¬ 
ipate  in  relationships  with  other  tokens.  Note  that  tokens  may 
be  defined  in  terms  of  other,  more  primitive  tokens,  as  in  the 
case  of  a  rectangle.  A  tokenaet  is  a  collection  (set)  of  tokens  of 
the  same  type  (e.g.  region  tokens).  Tokensets,  tokens,  token 
attributes,  and  relations  between  tokens  are  all  organized  into 
a  type  of  relational  database  called  the  “intermediate  symbolic 
representation”  (ISR).  The  ISR  allows  flexible  associative  access 
to  tokens  and  serves  as  the  communication  interface  between  the 
low-level  descriptive  processes  and  the  high-level  interpretive  pro¬ 
cesses  in  the  VISIONS  system.  In  general,  the  only  requirement 
for  placing  a  new  type  of  low-level  token  into  the  ISR  is  that  each 
primitive  element  of  that  data  type  must  have  a  symbolic  name 
(e.g.,  region-240,  surface-38,  corner-46)  and  a  non-empty  set  of 
attribute-value  pairs.  It  is  the  values  of  the  token  attributes, 
and,  as  we  shall  present  here,  the  relational  information  between 
tokens,  that  provide  the  basis  for  initial  interpretation  processes. 
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Figure  1.  The  VISIONS  System:  Processing  and  control 
across  multiple  levels  of  representation  are  depicted  in  this 
system  overview. 


3  Constraints  on  Tokens  of  a  Single  Type 

3.1  Constraint  Functions  on  Token  Attributes 

A  simple  type  of  knowledge  source  for  generating  hypothe¬ 
ses  of  object  class  labels  for  particular  regions  has  been  un¬ 
der  development  in  the  VISIONS  environment  for  some  time 
[BEL86,HAN87a]  (REY87a,RIS84,WEY83,WIL81).  The  general 
idea  is  to  develop  a  mapping  from  a  region  token  and  its  at¬ 
tributes  onto  an  object  label  hypothesis  for  the  region,  e.g.  ‘grass’. 
We  note  that  the  VISIONS  system  operates  primarily  in  the  out¬ 
door  scene  domain,  but  the  general  techniques  developed  below 
are  applicable  to  most  scene  domains  and  sensor  modalities.  The 
mapping  was  accomplished  by  defining  constraints  on  the  range 
of  an  attribute  from  a  sample  set  of  the  object  and  a  constraint 
function  which  mapped  from  the  region  attribute  into  a  weighted 
‘vote’  or  ‘score’  for  the  object  label  (see  Figure  2).  Compound 
constraints  were  defined  as  (possibly  recursive)  combinations  of 
the  output  of  a  set  of  these  ‘simple’  constraint  functions.  Region 
features  such  as  region  color,  texture,  shape,  size,  image  location, 
and  relative  location  to  other  objects  were  used.  More  recently, 
the  approach  has  been  extended  to  lines,  using  attributes  such 
ais  length,  orientation,  contrast,  width,  etc.  While  no  single  con¬ 
straint  on  the  features  of  a  region  or  line  can  ever  be  totally 
reliable,  the  combined  evidence  from  many  such  constraints  of¬ 
ten  imply  the  correct  interpretation  of  a  token;  for  example,  in  a 
rank  ordering  of  a  set  of  regions  on  the  basis  of  the  final  ‘score’, 
the  region-object  label  association  for  those  regions  near  the  top 
of  the  list  is  often  correct.  In  many  cases,  it  is  possible  to  de¬ 
fine  constraints  which  provide  evidence,  in  the  Dempster-Shafer 
sense,  for  and  against  the  semantically  relevant  concepts  repre¬ 
senting  the  domain  knowledge  [REY85,REY87a]. 

Rather  than  viewing  the  application  of  the  constraint  set 
through  the  constraint  function  as  a  classification  process  in  the 
pattern  recognition  sense,  the  rank-  ordered  output  can  be  used 
as  an  unreliable  set  of  hypotheses  and  used  to  trigger  focus  of  at¬ 
tention  mechanisms  in  an  artifical  intelligence  sense  {HAN78b]. 
They  are  used  by  other  more  complex  knowledge-based  processes 
in  a  hypothesize  and  verify  control  structure  [DRA87a,b,HAN87a] 
|WEY86|. 


Figure  2.  Structure  of  a  simple  constraint  function  as 
a  piecewise  linear  function  F  mapping  an  image  feature 
measurement  into  support  for  an  object  label  hypothesis. 
It  is  specified  via  6  points  =  1, ...  6}  . 


A  simple  constraint  function  (which  hereafter  will  often  be 
referred  to  simply  as  a  constraint)  is  a  function  F  applied  to  the 
attribute  (or  feature)  of  the  token  of  type  T.  Thus,  if 
the  attribute  of  line  tokens  (type  L)  is  length  and  Ljk  the 
length  of  the  line  token,  then  F{Ljk)  would  be  the  response  of 
the  constraint  function  when  applied  to  the  line  length  of  token 
j.  A  variety  of  forms  for  the  function  F  have  been  employed, 
with  no  appreciable  difference  in  the  results.  The  first  was  an 
extended  real- valued  piecewise-1  inear  function  F 

F{Tjfc)G  (10,11  U  VETO}  (1) 
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specified  by  six  points  in  the  feature  range  =  1,. .  .6  }  as 
shown  in  Figure  2  [RIS87,WEY83,WEY86].  Tlie  simplicity  of 
this  approach  is  that  F  in  this  form  could  be  compactly  stored 
as  a  6-tuple,  or  sets  of  6-tuples. 

Compound  constraints  are  a  hierarchical  collection  of  simple 
and  compound  constraints  with  an  arithmetic  or  logical  com¬ 
bination  function  for  collecting  the  individual  respon.sf’s  into  a 
single  response.  For  discussions  of  variations  on  compound  con¬ 
straints  see  |HAN87a,KIT86,RIS84,RIS87,WEY86].  In  some  ex¬ 
periments,  the  top-level  compound  constraint  for  an  object  was 
structured  as  a  combination  of  five  other  compound  constraints 
(as  shown  in  Figure  3)  to  represent  color,  texture,  size,  shape, 
and  location  constraints,  each  of  which  was  composed  of  a  set  of 
simple  constraints  [RIS87]. 


CRASS 


Figure  3:  The  structure  of  a  compound  constraint  for 
grass  showing  the  five  component  constraints  defined  on 
region  attributes. 


3.2  Relational  Similarity  Constraints  on  Tokens  of 
the  Same  Type 

When  dealing  with  unreliable  segmentation  processes,  forming 
aggregations  of  tokens  is  usually  necessary  since  the  set  of  tokens 
need  to  be  grouped  and  reorganized  in  order  to  match  an  object 
model  [DRA87b,HAN87a,HAN87b,REY87b).  The  basis  of  this 
grouping  usually  involves  not  only  the  attributes  of  tokens,  but 
also  the  relations  between  tokens.  The  constraiiits  described  in 
the  previous  section  are  unary,  since  they  accept  a  single  token 
attribute  as  input  and  return  a  value  that  can  be  viewed  as  a  con¬ 
fidence  or  rating  for  the  hypothesized  object.  The  highest  ranked 
of  these  hypotheses  can  serve  as  a  partial  (and  probably  errorful) 
interpretation  of  the  original  image.  However,  it  is  clear  that  con¬ 
straints  on  relationships  between  tokens  are  also  fundamental  to 
object  recognition.  They  can  be  handled  in  much  the  same  way 
by  defining  constraints  on  relational  measures  (a  function  which 
quantifies  a  particular  relation)  between  pairs  of  tokens.  In  this 
case,  the  response  of  a  constraint  function  specifies  the  degree  to 
which  the  constraint  on  a  relational  measure  is  satisfied. 

Let  us  consider  as  a  simple  example  a  line  token  set  and  a  re¬ 
lational  measure  defined  by  the  simple  absolute  difference  of  their 
orientation  attribute.  Given  a  specific  line  token  (e.g.  L-435),  all 
other  tokens  can  be  rank-ordered  relative  to  L-435  by  a  constraint 
function  applied  to  the  orientation  difference.  Figure  4  shows  two 
different  constraints  Fi  and  F2  for  processing  line  tokens  relative 
to  a  given  line  token.  Fi  gives  a  maximum  response  of  1  for  all 
tokens  that  are  within  10°  of  L-435,  a  linear  decreasing  response 
from  10°  to  30°  and  a  veto  response  beyond  45°.  The  effect  of 
applying  Fi  is  to  rank  equally  all  lines  that  are  very  similar  in 
orientation  to  that  of  L-435;  beyond  10°  they  are  ordered  based 
on  their  relative  orientation.  Lines  whose  relative  orientation  is 
greater  than  45°  away  from  that  of  L-435  are  excluded.  The  con¬ 
straint  embodied  in  Fg  results  in  the  selection  of  all  line  tokens 
that  are  within  5°  of  being  orthogonal  to  L-435.  In  effect  con¬ 
straint  F2  defines  a  relation  on  approximately  orthogonal  pairs  of 
lines  that  have  L-435  as  a  member;  the  relation  is  defined  to  have 
a  value  of  True  for  all  pairs  where  ^2(^435  ^;  Ly^jt)  =1.  In  the 


case  of  constraint  Fi,  the  relational  measure  for  those  lines  not 
vetoed  is  mapped  into  a  response  which  can  be  used  to  coarsely 
rank  order  the  line  pairs  in  terms  of  how  ‘paralleF  they  are.  Note 
that  applying  a  threshold  to  the  respnses  produces  a  true  rela¬ 
tion,  Either  of  the  subsets  resulting  from  application  of  these 
constraints  could  be  followed  by  a  token  attribute  constraint  for 


Absolute  Difference  in  Orientation 
(Degrees) 


Figure  4.  Example  Constraint  Functions  on  Relational 
Measures.  The  structure  of  two  constraint  functions  for 
relating  one  line  token  to  another  is  shown.  The  constraint 
is  applied  to  the  difference  in  orientation  of  two  line  tokens, 
(a)  Fi  equally  ranks  all  lines  whose  orientation  is  within 
±10°  of  the  given  line;  the  response  falls  off  as  a  function 
of  the  difference  in  orientation,  (b)  F2  equally  ranks  all 
line  tokens  that  are  within  ±5°  of  being  orthogonal  to  the 
given  token. 


ordering  or  filtering  the  remaining  lines  on  other  attributes  such 
as  location,  contrast,  length,  etc.  in  absolute  terms  or  relative  to 
Lr435.  In  the  next  section  we  show  how  this  general  idea  can  be 
extended  to  tokens  of  different  types,  resulting  in  a  fusion  of  the 
information  from  the  two  different  sources. 

4  Integrating  Representations  via  Construc¬ 
tion  of  Token  Aggregates 

The  fundamental  problem  that  is  being  addressed  in  this 
paper  is  the  integration  of  multiple  low-level  representations  into 
the  interpretation  process.  While  the  approach  presented  here 
offers  only  one  type  of  information  fusion  mechanism  and  deals 
with  only  some  of  the  most  general  levels  of  the  information  fu¬ 
sion  problem,  there  are  several  important  advantages.  First,  it 
offers  an  entirely  modular  and  natural  method  for  incorporat¬ 
ing  additional  processes  and  representations  as  a  vision  system 
undergoes  incremental  development;  in  particular,  existing  low- 
level  representations  do  not  have  to  be  modified  in  any  way.  Sec¬ 
ondly,  the  integration  is  accomplished  at  the  intermediate  group¬ 
ing  and/or  interpretation  levels  through  constraints  which  relate 
entities  in  the  independent  representations.  Since  there  is  no  di¬ 
rect  interaction  of  the  processes  which  initially  create  the  tokens, 
the  mistakes  of  one  low-level  process  will  not  affect  the  output 
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of  the  other  low-level  processes.  If  a  sufficient  body  of  consis¬ 
tent  information  exists  in  several  representations,  then  low-leyel 
mistakes  in  a  given  representation  may  be  detected  and  either  ig¬ 
nored  or  corrected,  as  opposed  to  integrating  partially  erroneous 
data  in  some  form,  such  as  a  least-squares  optimization  process. 
Third,  this  approach  is  an  extension  of  an  approach  applied  to  to¬ 
ken  attributes  that  has  already  proven  to  be  reasonably  effective 
on  very  complex  natural  scenes  [HAN86,HAN87a,RIS84].  Fi¬ 
nally,  the  techniques  can  be  used  as  part  of  general  intermediate 
grouping  processes.  The  grouping  can  be  viewed  as  knowledge- 
directed  (e.g.  via  a  model  of  an  object),  or  it  could  be  viewed  as  a 
data-directed  token  aggregation  process  whose  goal  is  to  extract 
interesting  structures  of  a  priori  importance  [REY87b]. 

4.1  The  Formation  of  Aggregations  of  Tokens 

Relational  constraints  are  defined  as  a  real-valued  function  on 
a  relational  measure  defined  over  a  token  set.  A  relational  mea¬ 
sure  M(Tj^kl  •  •  •  Tj„,k,n )  is  a  function  of  the  attributes  of  multiple 
tokens,  possibly  of  different  types.  As  we  have  already  discussed, 
for  tokens  of  the  same  type  there  is  an  implicit  binary  relational 
measure  in  that  the  same  scalar-valued  attribute  of  any  two  to¬ 
kens  can  be  compared  by  their  similarity  or  difference.  For  ex¬ 
ample,  given  a  specific  region,  similarity  relational  measures  be 

used  to  compute  the  distance  between  feature  centroids  as  well 
as  the  difference  in  mean  intensities  between  the  given  region  and 
all  other  regions.  Once  such  a  scalar  relational  measure  exists, 
a  constraint  function  can  be  applied  to  the  relational  measure 
to  produce  a  response  that  represents  the  degree  to  which  the 
relational  constraint  is  satisfied. 

In  order  to  compare  tokens  of  different  types  a  relational  mea¬ 
sure  must  be  defined  between  each  pair  of  token  types.  Therefore, 
if  new  sensors  are  added,  and  new  sensory  events  are  extracted, 
relational  measures  must  be  defined  between  the  existing  token 
types  and  the  new  token  types.  This  will  allow  information  from 
new  representations  to  be  integrated.  (In  this  paper,  only  binary 
measures  between  region  and  line  tokens  are  developed.)  The 
constraint  functions  on  relational  measures  can  then  be  applied 
to  sets  of  tokens  across  the  multiple  representations  in  order  to 
group  tokens  into  aggregations.  For  example,  relational  measures 
between  line  tokens  and  region  tokens  can  be  defined,  such  as  the 
degree  of  intersection  between  tokens;  a  constraint  on  this  rela¬ 
tional  measure  could  then  select,  for  each  region,  all  lines  that  are 
sufficiently  interior  to  the  region.  The  reader  should  note  that 
when  a  relational  constraint  is  used  to  filter  the  token  tuples,  into 
disjoint  sets,  the  result  is  a  relation. 

4.2  Region-Line  Relational  Measures 

We  now  present  a  specific  set  of  relational  measures  to  provide 
a  computational  method  of  relating  regions  and  lines.  Relational 
constraints  on  these  relational  measures  will  then  be  used  to  im¬ 
plement  the  following  relations  between  regions  and  lines: 

•  BOUNDING  lines  -  those  lines  associated  with  a  region 
boundary; 

•  INTERIOR  lines  -  those  lines  interior  to  a  region;  and 

•  OTHER  lines  -  those  lines  which  intersect  a  region,  but  are 
neither  bounding  nor  interior  to  the  region. 

The  relational  measures  chosen  are  based  on  the  intersection 
of  sets  of  pixels.  Only  lines  which  intersect  a  given  region  are  of 
interest.  We  will  represent  lines  by  the  subset  of  pixels  (called 
the  line-support-set,)  comprising  that  portion  of  the  intensity 
surface  that  led  to  the  extraction  of  the  line  [BUR86];  thus,  the 
line-support-sets  of  pixels  can  be  expected  to  overlap  the  regions 
that  they  bound  or  are  interior  to.  Consequently,  INTERSEC¬ 
TION  (defined  in  the  usual  way  on  pixel  subsets)  becomes  a  nat¬ 
ural  relation  which  can  be  used  as  a  filtering  constraint  to  select 
a  subset  of  tokens.  In  the  following  discussion,  however,  three 
other  relational  measures  will  be  defined  and  used  to  implement 


the  three  relations  mentioned  above:  “interior-line-percentage”, 
“region-perimeter-percentage” ,  and  “line-boundary-percentage” , 

The  first  relational  measure,  “interior-line-percentage” ,  is  the 
ratio  of  line  area  interior  to  the  region  to  total  line  area.  The 
interior-line-percentage  measure  discriminates  lines  that  are  en¬ 
tirely  INTERIOR  from  BOUNDING  lines,  whose  line-support 
set  will  lie  partially  outside  the  region.  An  INTERIOR  line  will 
have  a  value  of  100%  for  this  relational  measure,  indicating  that 
the  line-support-set  is  completely  contained  by  the  region.  An 
ideal  BOUNDING  line  with  a  symmetric  line-support-set  of  pix¬ 
els  lying  exactly  on  the  region  boundary  would  have  half  its  pixels 
in  the  region  and  a  value  of  50%  for  its  interior-line-percentage. 

The  other  twft  relational  measures  can  be  used  to  discrimi- 
nate  BOUNDING  Tines  from  INTERIOR  lines.  The  natural  du¬ 
ality  between  regions  and  their  boundary  lines  can  be  exploited 
in  a  straightforward  manner  to  indicate  how  much  of  a  region 
boundary  or  a  line  is  covered  by  the  other.  “Region-perimeter- 
percentage”  measures  the  fraction  of  a  region  boundary  made  up 
of  one  line  and  is  defined  to  be  the  ratio  of  the  intersection  of 
the  region  perimeter  pixels  and  the  line-support-set  to  the  length 
of  the  region  perimeter.  “Line-boundary-percentage”  measures 
the  fraction  of  a  line  contributing  to  the  region  boundary  and 
is  defined  to  be  the  ratio  of  the  intersection  of  the  line-support- 
set  and  the  region  perimeter  to  the  total  line  length  in  pixels. 
Ideally,  a  line  which  lies  approximately  on  a  region  boundary 
will  have  a  high  value  of  line-length-percentage  since  the  region 
boundary  will  cover  much  of  the  line.  The  same  is  true  of  region- 
perimeter-percentage  although  a  single  line  will  be  expected  to 
cover  a  smaller  portion  of  the  entire  region  boundary. 

4.3  Relational  Constraints 

Relational  constraints  are  used  as  the  final  step  in  the  for¬ 
mation  of  aggregations  from  multiple  representations.  The  rela¬ 
tional  measures  presented  in  the  preceding  sections  provide  the 
baisis  for  defining  these  relational  constraints. 

A  relational  constraint  function  for  lines  and  regions  can  be 
specified  for  each  relational  scalar  measure  that  has  been  de¬ 
fined  (in  the  same  manner  that  token  attribute  constraints  are 
defined).  Thus,  a  simple  constraint  can  be  specified  for  each 
of  interior-line-percentage,  line-length-percentage,  and  region-pe¬ 
rimeter-percentage  measures;  note  that  any  of  these  simple  con¬ 
straints  may  be  omitted.  A  combination  function  can  then  be 
defined  for  combining  the  output  of  the  set  of  simple  constraints 
into  a  compound  relational  constraint. 

The  form  of  the  function  that  combines  the  simple  constraints 
is  not  critical,  and  in  this  paper  we  will  use  the  same  simple 
piece  wise-1  inear  function  described  earlier  with  a  range  of 
{[0, 1]  U  V ETO}.  The  VETO  range(s)  serves  only  as  a  first  filter 

for  selecting  or  removing  candidates  for  processing,  in  the  sense 
that  the  vetoed  tokens  do  not  satisfy  the  constraint.  This  does 
in  fact,  define  a  relation  over  the  token  sets,  but  the  remaining 
non-vetoed  tokens  in  the  relation  still  have  the  graded  response 
from  the  constraint  function,  which  can  be  used  for  ranking  or 
further  filtering  of  token  pairs  to  produce  a  more  restricted  subset 
of  tokens. 

4.4  Controlling  the  Formation  of  Token  Aggrega¬ 
tions 

The  aggregation  of  tokens  via  relational  constraints  must,  of 
course,  contend  with  the  combinatorics  of  the  large  number  of 
image  tokens  whose  relationships  must  be  examined.  The  con¬ 
cept  ot  focus- of- attention  becomes  important  when  one  considers 
that  the  representations  being  used  typically  involve  2,000  to 
10,000  lines  and  200  to  1,000  regions.  Thus,  there  are  potentially 
400,000  to  10  million  line-region  pairs  which  could  be  related, 
and  these  numbers  become  much  larger  as  additional  representa¬ 
tions,  multiple  sensory  sources,  finer  image  resolutions,  or  larger 
images  are  considered. 

The  order  and  manner  in  which  attribute  constraints  and  rela¬ 
tional  constraints  are  applied  are  the  basis  of  the  control  strategy 
for  the  construction  of  token  aggregations.  Tokens  of  different 
types  can  be  aggregated  in  many  ways  and  one  seeks  to  avoid 
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the  combinatorics  of  computing  relations  between  large  token 
sets.  There  are  two  obvious  ways  in  which  the  constraint  func¬ 
tions  can  be  used  in  control.  A  constraint  on  the  cross  product 
of  the  token  sets  can  be  used  either  to  rank  order  the  set  of  token 
tuples  via  the  response  of  the  constraint  function,  or  to  filter  (i.e. 
select)  a  subset  of  the  token  tuples  for  further  processing.  Of 
course  the  responses  of  the  constraint  functions  that  are  used  for 
ordering  could  also  be  used  for  filtering  by  specifying  filtering  cri¬ 
teria  such  as  thresholds.  Example  control  strategies  for  limiting 
the  computation  are  outlined  in  [RIS87]. 

6  Results 

This  section  presents  the  results  of  forming  aggregations 
via  token  attribute  and  token  relational  constraints  applied  to 
suburban  house  scenes  and  road  scenes;  Figure  5  shows  a  repre¬ 
sentative  image  of  one  of  the  house  scenes  and  a  typical  region 
segmentation  and  line  description  for  the  image.  The  examples 
chosen  for  this  paper  involve  aggregations  of  tokens  that  serve  as 
texture  measures  and  aggregations  of  tokens  with  specific  shape 
properties. 

A  simple  texture  measure  based  on  line  density  can  be  com¬ 
puted  by  counting  the  number  of  lines  within  a  region  and  nor¬ 
malizing  by  the  region  size.  This  is  accomplished  by  forming 


Figure  5.  Results,  (a)  a  black  and  white  rendering  of  the 
original  color  image;  (b)  regions  produced  by  a  segmenta¬ 
tion  system  using  localized  histograms  followed  by  region 
merging;  (c)  straight  lines  produced  by  an  algorithm  which 
uses  similarity  of  gradient  orientation  as  the  primary  orga¬ 
nizational  feature. 


aggregations  of  regions  and  their  interior  lines.  A  filtering  con¬ 
straint  uses  the  interior-line-percentage  relational  measure  to  se¬ 
lect  only  those  lines  which  are  completely  (or  mostly)  interior  to 
the  region.  A  filter  is  defined  to  group  into  an  aggregation  those 
lines  associated  with  each  region  which  sufficiently  satisfy  both 
relational  and  attribute  constraints.  The  density  of  interior  lines 
in  each  aggregation  is  then  computed  as  an  attribute  of  this  new 
token  and  mapped  to  a  score  for  the  region  (which  can  also  be 
thought  of  as  a  score  for  the  region-line  aggregation). 

Figure  6a  shows  an  example  of  extracting  interior  lines  for 
regions  in  a  house  scene,  and  then  computing  the  interior  line 
density  of  these  regions  as  a  texture  measure  (see  Figure  6b). 
Some  objects,  notably  the  roof  of  a  house,  are  characterized  by 
short  horizontal  lines  (due  to  the  shingles)  interior  to  the  region. 
By  adding  the  additional  attribute  constraint  of  horizontal  orien¬ 
tation  on  the  interior  lines,  the  previous  result  can  be  extended  to 
focus  attention  on  the  house  roof  as  shown  in  Figure  6c ,d.  Addi¬ 
tional  constraints  on  line  length  and  line  contrast  can  be  defined 
to  extract  only  short,  horizontal  interior  lines  to  the  degree  that 
these  characteristics  of  the  expected  texture  element  are  known. 

The  roof  region  could  be  obtained  or  verified  in  another  way. 
The  line-boundary-percentage  relational  measure  could  be  used 
to  select  lines  which  lie  to  a  great  extent  on  the  boundary  of  the 
region  (see  Figure  7a).  A  line  attribute  rule  could  then  be  defined 
to  favor  long  lines  (see  Figure  7b).  The  lines  which  received  high 
scores  from  both  the  line  boundary  rule  and  the  line  length  rule 
(i.e.,  long  boundary  lines)  could  then  be  grouped  to  form  a  region¬ 
boundary  aggregation.  At  that  point  parallel  relations,  rectangle 
or  parallelogram  structures  could  be  identified. 

Another  simple  shape  measure  can  be  computed  by  deter¬ 
mining  if  a  region  is  bounded  by  a  pair  of  long  vertical  lines. 
As  Figure  8  shows,  the  process  is  useful  for  extracting  telephone 
poles  in  road  scenes.  The  filtering  relational  constraint  for  this 
measure  uses  line-boundary-percentage  to  select  only  those  lines 
which  lie  on  a  region  boundary;  an  attribute  constraint  then  se¬ 
lects  the  long  vertical  lines.  Relational  measures  can  be  defined 
to  form  aggregations  of  pairs  of  parallel  overlapping  lines  from 
the  long  vertical  boundary  lines. 

A  variety  of  more  complex  2D  shapes  can  be  matched  to 
lines  by  extracting  lines  that  bound,  regions.  The  techniques  pre¬ 
sented  here  are  only  a  part  of  more  complex  grouping  and  model 
matching  procedures  that  are  being  developed  in  other  research 
[HAN87a,REY87]. 
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Figure  6.  Example  of  Texture  Measure  for  Extracting  a 
House  Roof.  A  simple  texture  measure  computed  by  a  re¬ 
lational  constraint  bajsed  upon  the  density  of  lines  within  a 
region,  (a)  Lines  which  received  a  high  score  on  the  rela¬ 
tional  measure  of  interior-line-percentage  (i.e.  INTERIOR 
lines);  (b)  The  density  of  interior  lines  for  each  region  repre¬ 
sented  by  the  density  of  shading;  (c)  Horizontal  INTERIOR 
lines  for  the  roof  region;  (d)  Density  of  horizontal  interior 
lines. 


Figure  7.  Extracting  Roof  via  Long  Bounding  Lines.  Given 
a  possible  roof  region  the  long  bounding  lines  can  be  filtered 
to  find  a  roof  shape,  (a)  Lines  bounding  the  hypothesized 
roof  region,  and  (b)  Long  bounding  lines  which  can  be  the 
basis  of  forming  an  approximate  parallelogram  shape. 
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Figure  8.  Example  of  Extracting  Telephone  Poles  via  Ver¬ 
tical  Bounding  Lines,  (a)  The  line  pairs  formed  by  the  line 
constraint  of  bounding  vertical  lines,  (b)  The  relational 
measures  are  mapped  back  to  the  regions;  hatched  regions 
have  no  long  vertical  bounding  lines  and  are  vetoed. 


Figure  9  shows  a  set  of  rectangles  extracted  from  a  house 
scene.  In  this  case,  the  set  of  lines  intersecting  a  region  were 
filtered  to  extract  the  set  of  bounding  lines.  These  were  further 
filtered  on  the  basis  of  co-parallel,  collinear,  and  endpoint  coinci¬ 
dence  relations.  Pairs  of  ‘adjacent’  lines  were  then  filtered  on  the 
basis  of  constraints  on  their  relative  orientation  in  order  to  form 
corner  hypotheses  and  the  resulting  set  of  lines  were  matched 
to  a  rectangle  model.  As  the  figure  indicates  partial  matches  to 
the  rectangle  model  are  allowed.  The  matches  could  be  further 
restricted  if  one  is  seeking  dark  shutters  by  using  a  constraint 
on  the  intensity  of  regions,  although  this  was  not  done  in  this 
example. 

There  are  sometimes  serious  problems  with  constructing  ag¬ 
gregations  through  relational  measure  directly  computed  from 
initial  token  representations.  If  the  desired  primary  token  is  frag¬ 
mented,  whether  it  be  a  region  or  a  line,  then  the  expected  rela¬ 
tional  responses  might  be  distorted  significantly  because  some  of 
the  expected  token  attributes,  token  relations,  and  features  of  the 
extracted  aggregation  may  be  significantly  changed.  One  must 
balance  the  unreliability  of  extracting  useful  primary  tokens  by 
the  computational  savings  achieved  by  focussing  upon  the  subset 
of  secondary  tokens  that  satisfies  some  relational  constraints  with 
respect  to  the  filtered  primary  set.  To  the  degree  that  these  prob¬ 
lems  occur,  many  stages  of  hierarchical  token  aggregation  may 
be  necessary,  perhaps  using  more  complex  strategies  for  applying 
relational  measures  and  grouping  tokens. 

Let  us  consider  a  specific  real  example,  shown  in  Figure  10, 
where  both  representations  (e.g,  regions  and  lines)  would  have 
some  difficulty  in  directly  providing  the  basis  for  aggregations  of 


tokens  from  the  other  representations.  The  roof  region  in  Figure 
5b  is  fragmented  into  many  smaller  regions  (Figure  10a),  In  this 
case  no  region  will  get  full  benefit  of  the  bounding  roof  lines,  since 
each  region  is  only  bordered  by  a  subset  of  the  roof  lines;  Figure 
10b  depicts  the  lines  intersecting  the  largest  roof  region.  An  alter¬ 
nate  grouping  strategy  would  be  to  extract  a  set  of  roof  lines  first 
(which  may  be  a  difficult  task  in  itself),  then  the  set  of  regions 
that  are  bounded  by  these  lines  could  be  used  in  some  manner  to 
aggregate  regions.  The  line  information  if  properly  filtered  shows 
the  outline  of  the  roof  fairly  clearly  (Figure  10c),  but  the  initial 
set  of  lines  in  that  spatial  vicinity  provide  many  possible  aggre¬ 
gations.  The  process  of  grouping  lines  into  meaningful  geometric 
structures  is  a  non-trivial  problem  and  is  a  focus  of  continuing 
work  on  grouping  and  knowledge-directed  processing  in  our  re¬ 
search  group  [BOL87,HAN87a,HAN87b,REY87,WEI86,WEY86]. 
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Figure  9.  Extracting  Rectangular  Window  and  Shutter 
Hypotheses,  (a)  Horizontal  and  vertical  bounding  lines; 
(b)  The  regions  produced  by  a  relational  constraint  on  the 
extracted  bounding  lines. 

6  Conclusion 

The  use  of  constraints  on  relational  measures  between  tokens 
of  the  same  and  different  types  is  a  uniform,  straightforward  way 
of  combining  information  from  multiple  low-level  processes.  The 
techniques  developed  in  this  paper  allow  information  fusion  to 
take  place  during  the  interpretation  process  as  intermediate  to¬ 
kens  are  aggregated  via  object-dependent  constraints  on  token 
attributes  and  token  relations.  The  ideas  presented  here  can  be 
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Figure  10.  Example  of  difficulties  in  aggregating  tokens, 
(a)  The  roof  is  fragmented  in  the  region  representation;  (b) 
The  lines  intersecting  the  largest  roof  region  only  capture 
a  portion  of  the  relevant  line  information;  note  the  two  key 
missing  lines  at  the  upper  left  corner  of  the  roof,  (c)  A 
subset  of  the  full  set  of  lines  which,  if  they  could  somehow 
be  selected,  would  provide  the  appropriate  line  aggregation 
to  group  the  roof  regions,  and  allow  the  roof  outline  to  be 
completed  in  a  straightforward  manner  [WEY86]. 


used  to  build  hierarchical  aggregations;  for  example,  aggregates 
of  lines  could  be  formed  by  grouping  colinear  sets  of  lines  into 
new  longer  line  tokens  [e.g.,BOL87,REY87b,WEI86].  By  treat¬ 
ing  each  aggregation  as  a  new  token,  attributes  could  then  be 
computed  for  each  and  the  constraints  applied  recursively. 

The  types  of  information  that  could  be  added  include  sur¬ 
face  segmentations,  and  2D  and  3D  motion  and  depth  token  at¬ 
tributes.  Each  segmentation  or  low-level  process  would  create  a 
set  of  tokens  with  associated  attributes  which  could  be  added  to 
the  intermediate-level  representation.  These  new  tokens  could 
then  be  used  in  the  same  way  as  regions  and  lines  are  used  now. 


Each  new  token  type  would  require  the  definition  of  relational 
measures  between  tokens  of  different  types.  There  would  be  no 
other  major  modifications  to  the  system.  The  approach  over¬ 
laps  issues  and  techniques  in  the  areas  of  grouping  and  model 
matching.  To  the  degree  that  tokens  in  one  of  the  several  rep¬ 
resentations  do  not  exhibit  the  characteristics  that  provide  the 
basis  for  directly  extracting  the  desired  structures,  more  complex 
perceptual  organizing  processes  and  knowledge-based  strategies 
will  be  required.  Each  object  that  must  be  recognized  could  be 
defined  by  a  separate  model  and  control  strategy  for  aggregat¬ 
ing  the  different  token  types.  While  the  concepts  of  relational 
measures  and  relational  constraints  can  still  be  the  basis  of  these 
strategies,  many  stages  of  hierarchical  aggregations  may  be  re¬ 
quired.  In  such  cases  efficient  control  strategies  will  become  a 
major  issue.  Thus,  multiple  alternate  grouping  strategies  will 
probably  be  required  in  order  to  extract  and  utilize  information 
across  multiple  representations  in  a  generally  robust  and  effi¬ 
cient  manner.  In  the  VISIONS  system  [DRA87a,b,HAN87a],  the 
knowledge-based  schema  system  provides  flexible  mechanisms  for 
defining  and  applying  control  strategies.  The  mechanisms  de¬ 
scribed  in  this  paper  are  only  meant  to  serve  as  the  first  stage  of 
this  organizing  process. 
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1.0  INTRODUCTION 

Artificial  Intelligence  (AI)  techniques  are 
playing  an  increasingly  significant  role  in  computer 
applications.  Foremost  among  the  methods  of  AI  that  are 
finding  wide-spread  use  are  those  that  underlie 
knowledge-based  systems.  These  are  programs  designed 
so  that  the  knowledge  needed  to  solve  problems  can  be 
expressed  separately  and  manipulated  independently  of 
the  part  of  the  program  that  applies  the  knowledge. 

This  separation  of  the  knowledge  from  the  mechanisms 
that  apply  it  has  proven  to  be  an  exceptionally  powerful 
programming  technique.  As  a  result,  knowledge-based 
technology  will  be  an  increasingly  important  tool  in  the 
application  of  computing  machinery  to  complex  and 
heretofore  unmanageable  problems. 

A  number  of  systems  currently  under  exploratory 
development  take  knowledge-based  approaches  to  the 
task  of  data  fusion  in  the  tactical  intelligence  domain. 
Included  among  the  activities  addressed  by  these 
systems  are: 

1.  to  correlate  raw  intelligence  reports  to  lower 
echelon  units  (refs.  1,  2,  and  3); 

2.  to  aggregate  lower  level  units  into  structured, 
higher-level  echelon  units  (refs.  4  and  5); 

3.  to  predict  activities  and  motions  for  military  units 
(refs.  6  and  7); 

4.  to  identify  strategies,  objectives,  and  decision 
points  for  (usually  higher-level)  units  (refs.  8,  9, 
and  10) 

These  developments  are  expected  to  bear  fruit  in 
the  reasonably  near  future.  They  will  be  the  under¬ 
pinnings  of  decision  aids  for  intelligence  staffs  which 
will  help  construct  and  evaluate  hypotheses  about  the 
enemy's  dispositions  and  probable  courses  of  action. 

Before  such  systems  can  be  fielded,  however,  a 
problem  of  particular  importance  to  the  intelligence 
community  must  be  addressed:  the  knowledge  domain 
is  extremely  volatile.  This  point  strongly  impacts  the 
design  issues  of  a  fully  adequate  decision  aid  system  for 
intelligence  staff.  Not  only  will  the  crucial  knowledge 
components  of  such  systems  refer  to  uncertain  elements 
of  the  enemy's  behavior  such  as  doctrine  and 
tendencies,  but  the  nature  of  those  elements  is  that  they 
arc  constantly  changing  as  the  enemy  gains  experience 
and  adopts  new  tactics  and  strategies.  These  factors 
make  it  extremely  important  to  explore  methods  by 
which  knowledge  may  be  dynamically  evaluated  for 
currency,  and  system's  behavior  adapted  in  the  face  of 
knowledge  failures. 


measure  of  system  performance  should  apply  to 
individual  rules  in  the  system  since  it  is  modifications  to 
these  that  give  rise  to  modifications  of  system  behavior. 

This  paper  presents  research  in  progress  in 
machine  learning  as  applied  to  sensor  data  fusion.  In  it 
we  discuss  a  dynamic  approach  to  the  measurement  rule 
performance  in  data  fusion  systems  for  tactical 
intelligence.  We  then  extend  this  approach  to  the 
problem  of  assessing  the  performance  of  the  system's 
information  sources.  This  capability  is  expected  to  be  of 
use  in  two  areas.  It  will  increase  the  capacity  of 
automated  infor-mation  integration  systems  to  deliver 
high-quality  support  to  intelligence  staff  and  it  can 
serve  as  input  to  collection  management  staff  in  the 

The  paper  is  organized  into  six  sections.  In 
Section  2.0  we  briefly  describe  the  operation  of  rule- 
based  systems.  We  illustrate  the  ability  of  an  existing 
algorithm,  the  bucket  brigade,  to  lead  to  a  solution  of  the 
credit  assign-ment  problem  in  Section  3.0.  In  Section 
4.0  the  method  is  extended  to  environments  in  which 
there  is  no  access  to  ground  truth.  We  discuss 
extensions  which  make  similar  evaluation  possible  on 
sources  of  information  in  Section  5,0.  Section  6.0 


Ui  RULE-BASED  SYSTEMS 

The  most  common  method  of  representing 
knowledge  inside  a  computer  is  as  a  set  of  rules.  The 
rules  f  are  if-then"  statements  describing  the  conditions 
(the  "if  part,  or  antecedent,  of  the  rule)  under  which 
the  rule  could  be  applied,  and  the  action  (the  "then" 
part,  or  consequent  of  the  rule)  to  be  taken  when  the 
rule  is  applied.  For  example,  a  knowledge  base  designed 
to  detect  artillery  and  armored  units  from  sensor  reports 
might  consist  of  the  following  rules: 

RULEl:  IF  there  is  a  report  of  a  shooter  at  a  time 

and  location  and  there  is  a  report  of  a 
mover  at  the  same  time  and  location 

THEN 

there  is  a  tank  at  the  time  and  location 

RULE 2:  IF  there  is  a  report  of  a  shooter  at  a  time 

and  location 

THEN 

there  is  artillery  at  the  time  and  location 

RULE3:  IF  there  is  a  communications  report  at  a 

time  and  location 


Machine  learning  is  the  branch  of  artificial 
intelligence  that  encompasses  adaptive  behavior  of 
knoweldge-based  systems.  It  is  defined  as  the  automated 
modification  of  a  system's  behavior  to  improve  its 
performance.  For  any  realistic  system,  the  main 
obstacles  to  machine  learning  are  the  problems  of  (1) 
effectively  evaluating  the  performance  of  those 
components  of  a  system  open  to  modification,  and  (2) 
finding  modifications  of  those  components  which  will 
improve  performance.  Thus  for  rule-based  systems  the 


THEN 

there  is  a  radio  at  the  time  and  location 

RULE4:  IF  there  is  a  tank  at  a  time  and  location 

and  there  is  a  radio  at  the  same  time  and 
location 

THEN 

there  is  an  armored  unit  at  the  time  and 
location 
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RULES:  IF  there  is  artillery  at  a  time  and 

location  and  there  is  a  radio  at  the  same 
time  and  location 

THEN 

there  is  an  artillery  unit  at  the  time  and 
location 

These  rules  are  over-simplified  examples  of  the 
rules  that  would  appear  in  an  operational  system.  They 
are  intended  for  pedagogical  purposes  only  and  in  this 
case  model  an  environment  in  which  "armored  units" 
move,  shoot,  and  communicate  all  the  time,  while 
"artillery  units"  just  shoot  and  communicate. 

For  tasks  of  even  moderate  complexity,  many 
rules  may  be  candidates  for  application  at  a  given  time. 
Notice,  for  instance,  that  Rule2  will  have  its  antecedent 
matched  any  time  RuleTs  match.  An  important  aspect  of 
the  intelligent  behavior  of  a  system  involves  the 
selection  of  the  rule  from  among  those  matched  that  is 
to  be  fired.  The  firing  of  a  rule  may  in  turn  change  the 
set  of  rules  whose  antecedents  match.  The  following 
example  will  illustrate  the  process.  Suppose  we  receive 

(Shooter  reported  at  Timel  and 
Location] ) 

(Moyer  reported  at  Timel  and  Location!) 
(Communication  reported  at  Timel  and 
Location! ) 

Rules  1,  2,  and  3  have  their  antecedents  matched 
by  the  reports.  By  the  conventional  account  of  conflict 
resolution  for  rule-based  systems  (ref.  11)  the  rule  with 
the  most  antecedents,  Rulel  in  this  case,  is  chosen  for 
firing.  In  firing,  it  uses  the  shooter  and  mover  reports 
to  establish  the  existence  of  a  tank  at  Locationl,  Timel. 
Rule2  is  removed  from  consideration  since  the  shooter 
report  that  originally  made  its  antecedent  match  was 
used  by  Rulel.  Rule3  still  matches  and  it  is  fired,  using 
the  communications  report  to  establish  a  radio  at 
Locationl,  Timel.  The  system  now  holds  that: 

(There  is  a  tank  at  Timel  and  Locationl) 
(There  is  a  radio  at  Timel  and  Locationl) 

Rule4  now  matches  and  is  fired.  It  establishes: 

(There  is  an  armored  unit  at  Timel  and 
Locationl ) 

The  inference  process  described  is  typical  of 
contemporary  knowledge-based  systems.  The  main 
difference  of  actual  systems  and  the  process  described 
above  is  in  the  amount  and  the  types  of  knowledge 
brought  to  bear  on  a  given  problem.  For  instance, 
actual  systems  are  likely  to  reason  from  a  set  of 
plausible  missions  the  enemy  may  be  conducting  as  well 
as  positions  of  equiptments  and  records  of  activities. 


3.  ft  CREDIT  ASSIGNMENT 

One  of  the  ways  in  which  the  knowledge-based 
system  described  above  is  typical  is  that  it  depends  on 
the  knowledge  base  being  static.  Once  it  is  written  it 
will  follow  the  same  inference  paths  given  siniilar 
infor-mation.  It  has  no  way  to  modify  its  behavior.  If  it 
turned  out,  for  instance,  that  there  were  no  armored 
unit  at  Locationl  at  Timel,  it  would  take  no  cognizance 
of  its  error.  As  mentioned  above,  however,  the  tactical 
intelligence  domain  is  highly  volatile  in  terms  of  the 
information  and  knowledge  of  the  enemy.  In  this 
section  we  describe  an  elaboration  of  the  inference 
procedure  which  evaluates  the  knowledge  in  the  system 

The  major  difficulty  of  effectively  evaluating 
rules  in  a  system  is  the  proper  apportionment  of  credit 
for  successful  rule  actions  (or  assignment  of  blame  for 
erroneous  ones)  (refs.  12-13).  The  problem  is  that  any  of 
the  rules  leading  to  the  action  may  merit  the  most 
responsibility  or  culpability  for  the  action.  Consider  the 
example  above.  If  no  armored  unit  is  subsequently 
found  at  the  specified  time  and  location,  any  one  of 
Rulel,  Rulc2,  Rule4,  or  any  of  their  combinations  might 
be  to  blame  for  the  error.  For  complex  knowledge 
systems,  this  becomes  an  extremely  difficult  problem. 

3.1  The  Bucket  Brigade _ Algorithm 

We  have  been  exploring  an  approach  to  this 
problem  based  on  the  bucket  brigade  algorithm  (refs. 
14-15).  The  bucket  brigade  was  developed  originally  in 
the  context  of  work  on  population  genetics  to  simulate 
the  fitness  of  individuals  in  a  population  to  reproduce 
(ref.  16).  In  our  work,  it  functions  by  simulating  an 
artificial  economy  among  the  rules  in  the  system. 
Through  a  competitive,  bidding-and-payoff  process,  the 
rules  accumulate  or  lose  capital  based  upon  the 
contribution  to  useful  system  behavior.  The  strength  of 
a  rule  as  measured  by  its  capital,  called  its  wealth,  is  thus 
a  measure  of  its  utility.  In  the  classic  bucket  brigade, 
utility  derives  ultimately  from  a  scheme  of  rewards 
administered  to  the  system  for  "good"  actions.  These 
rewards  arc  called  "environmental  payoffs"  because 
they  derive  from  the  direct  interaction  of  the  system 
with  the  environment.  The  basic  process  is  explained 
below,  while  Section  4.0  contains  a  description  of  an 
"unsupervised"  implemen-tation  of  the  algorithm  that 
does  not  depend  on  direct  intervention  ("supervision") 

We  illustrate  the  bucket  brigade  using  our 
previous  example.  Each  rule  is  given  an  initial  amount 
of  capital.  We  distribute  the  capital  carefully  in  this 
example  so  we  may  demonstrate  interesting  features  of 
the  algorithm.  However,  it  will  be  seen  that  these  initial 
wealths  are  dynamically  adjusted;  their  initial 
distribution  doesn't  matter  in  the  long  run.  Let  the 

RULEl:  8 

RULE2:  7 

RULE3:  7 

RULE4:  16 

RULES:  16 
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intent  is  to  model  an  artillery  unit  at  Location  1  plus  a 
bogus  mover  report  from  the  same  location.  Let  the  new 
batch  of  reports  come  in  looking  as  follows: 

(Shooter  reported  at  Time2  and  Location!  ) 
(Mover  reported  at  Time!  and  Location!  ) 
(Communication  reported  at  Time!  and 
Location!  ) 

When  these  are  placed  in  working  memory,  Rules 
1,  2,  and  3  again  have  their  antecedants  matched, 
resulting  in  the  following  bids: 

RULEl:  4 

RULE2:  3.5 

RULE3:  3.75 

Again,  Rulcl  is  chosen  to  fire  from  among  its 
conflict  set,  followed  by  Rule2.  They  establish  the 
conditions  for  Rule4  to  fire,  but  because  its  wealth  is 
now  only  8,  its  bid  is  down  to  4.  It  wins  the  bid,  again 
it  has  no  competitors,  and  pays  out  2  each  to  Rules  1  and 
3.  The  wealths  of  the  rules  in  the  rule  base  are  now: 

RULEl:  6 

RULE2:  7 

RULE3:  5.75 

RULE4:  4 

RULES:  16 

Rulel’s  wealth  has  started  to  decrease.  After  the 
second  batch  of  reports,  the  failure  of  the  inference 
chain  to  establish  correct  results  affects  the  wealth  of 
the  contributing  rules  one  step  back  in  the  inference 
chain.  This  in  turn  affects  the  ability  of  those  rules  to 
compete  in  the  bidding  process.  Observe:  when  the  next 
batch  of  reports  comes  in  (identical  with  the  first  two, 
but  for  the  time  being  Time3),  the  bids  of  Rules  1,  2,  and 
3  are  respectively  3.  3.5,  and  2.875.  This  time  Rule2  wins 
the  bid,  establishing  artillery  at  Location  1,  Time3.  Rulel 
is  taken  out  of  the  conflict  set,  and  Rule3  is  fired.  When 
that  happens.  RuleS's  antecedants  are  matched.  It  bids  8, 
is  fired,  and  establishes  an  artillery  unit  at  Location  1 
Time3.  This  is  what  we  have  been  waiting  for,  and  we 
reward  RuleS  with  some  arbitrary  sum  related  to  the 
importance  of  the  result.  Here,  RuleS  receives  an 
"environmental  payoff"  of  16. 

The  effects  of  the  incorrect  action  have  now  been 
overcome  by  the  work  of  the  competitive  algorithm. 
Continued  reports  will  strengthen  the  inference  chains 
that  led  to  artillery  unit"  until  a  stable  set  of  wealths  is 
achieved  at  each  rule. 


O  unsupervised  T.FARmMq 

The  algorithm  as  presented  so  far  is  a  supervised 
algorithm.  It  works  only  because  an  omniscient 
researcher  distributes  environmental  payoffs  on 
favored  actions  by  the  system.  This  is  unacceptable  for 
our  application  of  interest  since  no  access  to  absolute 
know-ledge  is  available.  Even  a  direct  report  on  a  state 


The  algorithm  works  by  modifying  the  strategy 
by  which  the  system  decides  which  of  a  set  of  matching 
rules  to  fire.  The  set  of  rules  matching  at  a  given  time  is 
called  the  conflict  set,  and  the  method  for  choosing 
among  them  is  called  the  conflict-resolution  strategy. 
The  bucket  brigade  implements  an  "economic" 
competition  based  on  bidding  behavior  by  the  rules.  The 
new  view  of  conflict  resolution  is  that  each  rule 
matched  is  competing  for  the  privilege  to  establish  its 
consequent.  The  bid  is  based  on  how  well  the  rule 
matched  the  environment  (its  specificity),  the  wealth  of 
the  rule,  and  a  factor  c,  where  0  <  c  <=  1,  which  restricts 
the  percentage  of  its  wealth  a  rule  may  risk  in  any  one 
bidding  cycle.  Rules  all  use  the  following  function  to 

Bid  =  (specificity)  x  (c)  x  (wealth) 

The  specificity  is  1.0  if  the  environment  matches 
the  rule's  left-hand  side  exactly  (versus  partial  or  more 
general  matches).  The  bid  is  a  "closed"  bid;  no  rule  bases 
its  bid  on  the  bidding  behavior  of  other  rules.  The 
highest  bid  in  the  conflict  set  wins.  For  our  purposes, 
c  —  0.5  and  specificity  =  1.0,  thus  the  competition  depends 
strictly  on  relative  rule  wealths.  When  the  three 
reports  come  into  the  (modified)  system,  Rules  1,  2,  and  3 
make  bids  respectively  as  follows: 

RULEl:  4 

RULE2:  3.5 

RULE3:  3.5 

As  before,  Rulel  is  chosen  to  fire  from  among  its 
conflict  set  competitors.  This  time,  however,  some  new 
events  take  place.  Rulel  must  pay  out,  from  its  capital, 
the  amount  of  the  bid.  After  Rulel  fires,  its  wealth  is 
decre-mented  to  4;  it  is  at  risk  by  the  amount  of  its  bid. 
Also,  the  result  of  Rulel's  firing  is  tagged  with  the  fact 
that  Rulel  established  it.  Rule2  is  taken  out  of  the 
conflict  set,  leaving  only  Rule3.  When  it  fires,  its 

When  Rule4's  antecedants  are  matched  it  enters 
the  conflict  set  and  makes  a  bid  of  8.  It  wins  the  bid  (by 
virtue  of  being  in  the  conflict  set  alone)  and  must  pay 
out  its  bid  to  establish  its  result  --  an  armored  unit  at 
Locationl  at  Timel.  The  crux  of  the  bucket  brigade 
method  for  credit  assignment  is  that  the  bid  that  Rule4 
pays  out  is  paid  to  Rules  1  and  2,  so  their  respective 
wealths  are  incremented  to  8  and  7.5.  Rules  1  and  2  have 
recouped  their  temporary  losses;  their  investment  has 

Had  there  been  an  armored  unit  at  the  specified 
location,  the  bucket  brigade,  to  work  correctly,  would 
require  that  a  reward  be  given  to  Rule4  for  posting  a 
correct  result.  In  this  case,  however,  we  are  postulating 
that  there  is  no  armored  unit  at  the  specified  location,  so 
no  reward  is  given  to  Rule4,  and  it  loses  the  capital  it 
spent  on  the  bidding  process.  It  is  the  first  to  feel  the 
effects  of  error. 

2*2 — A^daptive  Behavior  bv  Comnetitive  Confiirt 
RESOLUTION 

The  fact  that  there  was  no  armored  unit  at 
Locationl  is  reflected  in  the  new  set  of  reports.  The 
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of  affairs  is  qualified  by  an  assessment  of  the  position  of 
the  source  to  know  the  state  of  affairs,  the  source  s 
competence,  and  the  source's  reliability.  In  this  section 
we  extend  the  system  to  operate  in  unsupervised 
environments. 


In  order  to  achieve  unsupervised  learning,  the 
learning  system  must  adjust  its  behavior  strictly  on  the 
basis  of  its  “sensory"  input.  In  our  example,  this  is  the 
report  stream  represented  by  the  mover,  shooter,  and 
radio  reports.  We  impose  the  condition  upon  the  system 
that  a  report  (either  predictive  or  retrodictive)  must  be 
issued  with  the  establishment  of  every  abstract  concept 
(i  e ,  a  concept  introduced  by  inference  from  reports, 
such  as  “radio").  That  is,  all  concepts  in  the  system  must 
have  observational  consequences  at  a  time  other  than 
that  at  which  it  is  must  be  established  without  its  being 
of  reports  at  some  time  other  than  that  at  which  it  is 
"observed". 


We  implement  this  principle  through  the 
following  rules: 


RULE6:  IF  there  is  a  tank  at  a  time  and  location 

THEN 

there  will  be  a  report  of  a  shooter  and  a 
report  of  a  mover 
at  that  time  and  location 


These  rules  are  more  or  less  the  “inverse"  of 
Rulesl-5,  above,  taking  the  conceptual  entities 
established  by  them  back  down  to  a  predicted  set  of 
sensor  reports.  (The  "following  time  and  new  location" 
referred  to  in  Rule9  is  just  shorthand  for  the  projected 
times  and  places  that  would  be  computed  in  a  real 
system).  The  new  rules  need  not  have  been  such  direct 

inverses.  As  mentioned  before,  in  actual  systems  the 

analysis  of  expectations  is  expected  to  hinge  on  an 
analysis  of  the  goals  and  the  plans  to  achieve  those  goals 
of  the  objects  of  interest  (e.g.,  armored-unit’s  missions). 

The  operation  of  this  expanded  rule  set  is  just  as 
before,  except  the  chaining  continues  through  the 
newly  added  rules  and  results  in  a  set  of  propositions 
about  reports  at  other  times.  The  appropriate  place  for 
the  evaluation  of  the  system’s  performance  has  been 
temporally  deferred  through  these  new  rules  to  the  next 
time  step  and  logically  referred  to  assertions  about  the 
report  stream.  The  possibility  for  unsupervised 
processing  is  now  seen  to  be  much  closer.  This  is 
because  the  new  set  of  assertions  is  easy  to  assess:  we 
simply  compare  the  predictions  to  the  report  stream  and 
reward  rules  whose  predictions  are  correct. 


Given  the  arrangement  described  above, 
construction  of  an  unsupervised  system  can  be 
elegantly  achieved  by  the  addition  of  a  meta-level  rule 
(a  rule  that  mentions  rules  in  its  clauses)  such  as: 


RULE?:  IF  there  is  artillery  at  a  time  and 

location 

THEN  . 

there  will  he  a  report  of  a  shooter  at  that 
time  and  location 

RULES:  IF  there  is  a  radio  at  a  time  and  location 

THEN 

there  will  be  a  communications  report 
at  that  time  and  location 

RULE9:  IF  there  is  an  armored  unit  at  a  time  and 

location 

THEN 

there  will  be  a  tank  and  a  radio 
at  the  following  time  in  a  new  location 

RULEIO:  IF  there  is  an  artillery  unit  at  a  time  and 

location 

THEN 

there  will  be  artillery  and  a  radio  at  that 
time  and  location 


ENV:  IF  a  rule  predicts  an  occurence 

and  the  occurence  is  reported 
THEN 

increase  the  rule's  wealth 

The  metarule  implements  environmental  payoff 
of  correct  predictions  by  the  explicit  action  of 
rewarding  the  rules  responsible  for  the  predictions. 
Notice  that  the  rule  is  a  second-order  rule  in  two  ways: 
(1)  it  mentions  rules  in  its  antecedants  and  consequents, 
and  (2)  the  action  it  takes  has  a  direct  effect  on  a 
characteristic  of  the  rule,  its  wealth,  which  is  not  a 
domain-level  but  system-level  characteristic  of  the  rule. 

A  useful  elaboration  of  this  rule  computes  a  share 
of  the  system’s  overall  payoff  budget  that  the  rule  is  to 
receive: 


**  A  rule  that  predicts  a  report  where  there  is  no  sensor 
coverage  should  not  be  viewed  as  having  generated  a 
failed  prediction.  To  ensure  stable  operation,  the  system 
should  pay  such  a  rule  the  amount  of  its  bid  so  that  there 
is  neither  gain  nor  loss  to  the  rule’s  wealth  in  such 
circumstances. 
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ENV2:  IF  a  rule  predicts  an  occurence 

and  the  occurence  is  reported 

THEN 

increase  the  rule's  wealth 
by  a  specific  proportion  of  the 
total  payoff  budget 

This  rule  has  the  virtue  of  implementing  a  closed 
economy  (in  terms  of  the  overal  wealth  in  the  system). 
We  are  pursuing  analysis  of  such  closed  systems  to 
determine  what  convergence  and  stability  results  hold 

Another  way  to  achieve  payoff  strictly  through 
the  action  of  rules  is  to  simplify  the  environmental  rule 
as  follows: 

ENV3:  IF  a  rule  predicts  an  occurence 

and  the  occurence  is  reported 

THEN 

report  the  occurence  with  ENV  as 
the  source 

This  rule  demands  that  when  a  report  is  entered 
in  the  system,  its  source  is  given  as  ENV3.  ENV3  is 
expected  to  be  given  a  large  share  of  wealth.  When  it  is 
invoked  it  pays  itself  part  of  its  bid  (as  the  source  of  its 
second  clause),  and  pays  the  rule  that  made  the  correct 
prediction  the  other  part.  This  depletion  of  its  wealth  is 
made  up  for  by  the  fact  that  when  low-level  sensor 
processing  rules  (e.g.,  Rules  1,  2,  and  3,  above)  use 
reports,  they  pay  their  bids  to  ENV3.  In  this  way,  ENV3 
implements  the  desired  functionality  strictly  within  the 
competitive  bidding  process  described  above. 

Each  of  the  metarules  described  implements 
unsupervised  learning  by  driving  the  system  towards 
sets  of  rules  that  make  correct  predictions.  The  result  is 
exactly  as  in  the  supervised  case  described  above,  except 
that  now  a  rule's  wealth  is  proportional  to  its 
contribution  to  the  system’s  overall  predictive  power. 

On  the  view  implemented  here,  verifiable  predictions 
are  the  only  ultimate  tests  of  correctness  available  to  the 
system.  In  this  system,  utility  is  a  completely  internally 
assessed  function  of  the  system's  ability  to  generate 
correct  predictions  about  future  enemy  activities.  It 
assesses  its  knowledge  exclusively  in  those  terms 
without  a  require-ment  for  access  or  comparison  to 


Recall  that  the  report  stream  is  intended  to  reflect 
the  situation  where  an  artillery  unit  is  stationed  at  some 
location,  and  a  sensor  is  generating  erroneous  or  bogus 
mover  reports  to  the  rule-based  system.  The  system  first 
believes  it  is  in  the  presence  of  an  armored  unit,  but 
soon  begins  to  doubt  its  results  since  they  consistently 
go  unconfirmed.  The  system  finally  decides  it  is  looking 
at  an  artillery  unit.  Thus  the  system  achieves  some  level 
of  adaptation  in  the  face  of  an  unexpected  situation. 

That  is  not  enough.  A  more  complete  system  would  not 
only  assess  the  rules  leading  to  the  system's  results,  but 
also  assess  the  sources  involved  in  its  processing,  using 


the  source  assessments,  for  instance,  in  re-evaluating 
its  hypotheses  or  in  collection  management.  We  are 
exploring  approaches  to  sensor  evaluation  by  the 
removal  of  the  all-encompassing  "ENV'-type  metarules 
and  the  introduction  of  an  "environmental"  rule  for 
each  external  source  of  information  available  to  the 
system.  Such  rules  would  look  as  follows: 

Shooters:  IF  a  rule  predicts  a  shooting 
activity 

and  the  shooting  activity  is 
reported 

THEN 

report  the  shooting  activity 

with  "Shooters"  as  the  source 

Com  ms:  IF  a  rule  predicts  a 

communications  activity 
and  the  communications  activity 
is  reported 

THEN 

report  the  shooting  activity 

with  "Comms"  as  the  source 

Movers:  IF  a  rule  predicts  movement-type 

activity  and  the  motion  is  reported 

THEN 

report  the  motion 
.  with  "Movers"  as  the  source 

Just  as  in  the  case  of  ENV3,  the  report  sources, 
now  defined  as  report-generating  rules,  accumulate 

wealth  when  sensor-processing  rules  use  the  source's 

reports  and  disburse  payoffs  when  a  rule  correctly 
predicts  a  report  emanating  from  the  source. 

Using  the  same  sequence  of  reports  as  before,  the 
effects  of  differentially  accumulating  wealth  according 
to  these  three  sources  would  be  for  each  to  accumulate 
wealth  as  their  reports  are  used  by  Rules  1,  2,  and  3. 

This  would  continue  uninterrupted  until  the  system 
decided  it  was  looking  at  an  artillery  unit,  at  which  time 
correct  pre-dictions  would  start  being  made.  Once 

correct  predictions  begin,  the  sources,  in  this  case 
Shooters  and  Comms,  would  dispense  payoffs  to  Rules  7 
and  8  analogous  to  how  ENV3  would  have  dispensed 
payoff  for  correct  predictions.  Movers  would  keep  its 
wealth  intact.  However,  in  this  case  Movers  also  would 
cease  to  play  a  part  in  the  processing,  since  Rulel, 
which  was  the  only  rule  using  mover  reports,  starts 
losing  the  bid  for  the  shooter  reports  to  Rule2  (recalling 
the  example  at  the  beginning  of  Section  3.0),  The  source 
of  mover  reports  has  effectively  been  isolated  from 


What  are  we  to  make  of  the  wealths  associated 
with  sources?  Movers'  wealth  was  equal  to  that  of 
Shooters  and  Comms  and  actually  became  greater  than 
theirs  once  they  began  providing  environmental  payoff 
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to  the  predictive  rules.  Therefore  we  cannot  interpret  a 
sensor's  wealth  as  a  direct  figure  of  merit.  All  the  less  so 
since  we  purposely  set  the  stage  with  a  false  sensor 
stream  from  Movers!  Is  wealth  a  direct  utility  measure? 
This  doesn’t  seem  appropriate  either,  since  the  most 
useful  thing  about  mover  reports  in  this  report  stream 
is  to  find  out  that  you  shouldn’t  use  them. 

When  a  source  fails,  there  seems  to  be  an  inverse 
relation  between  the  wealth  of  source  and  its  ultimate 
predictive  power.  Once  predictions  cease  to  be  possible 
on  the  source,  its  wealth  strictly  accumulates  until  it  is 
isolated.  It  appears  that  the  wealth  accumulated  up  to  the 
isolation  point  of  a  source  is  simply  a  lost  investment  by 
the  system.  Possibly  rules  that  make  predictions  about 
erroneous  or  misleading  sources  might  lead  to  correct 
predictions,  freeing  this  wealth  for  the  system,  but  we 
have  not  yet  experimented  with  this. 

As  we  noted  in  the  example  above,  inference 
paths  that  lead  to  failed  predictions  atrophy.  In  time 
other  hypotheses  are  tried,  and  if  some  of  these  succeed 
they  will  isolate  the  erroneous  source  from  further  use 
by  the  system.  If  this  does  not  happen,  then  either  the 
inference  paths  effected  continue  to  loose  wealth  or 
they  form  a  self-supporting  cycle  of  predictions  in  the 
system.  The  former  is  easily  detectable,  but  it  is 
computationally  impractical  to  analyse  the  inference 
for  such  cycles.  This  suggests  the  importance  of  multi- 
a  corrective  to  this  type  of  problem. 

We  have  presented  an  approach  to  the  automated 
evaluation  of  hypotheses  and  information  sources  for 
rule-based  systems.  This  work  is  being  pursued  at  The 
MITRE  Corporation  in  the  context  of  a  machine  learning 
project  (refs.  17-19).  The  dynamic  rule  evaluation  is  to 
guide  the  automated  rule  modification  module  of  a 
system  which  will  suggest  new  hypotheses  about  the 
domain  to  the  user.  We  believe  it  will  prove  equally 
useful  in  interactive  rule  editing,  where  we  expect  the 
evaluation  measures  to  guide  the  user  to  areas  of  the 
rule  base  needing  modification,  and  in  decision  aids  in 
the  area  of  collection  management,  where  a  measure  of 
efficacy  for  sources  in  the  interpretation  of  a  situation 
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Pvictino  TflPtiral  031  Centers  are  limited  in  their  ability  to  process  (accept,  analyze,  correlate,  fuse,  display,  and  manage)  the  volumi- 
result,  will  require  more  automation  and  processing  sophistication. 

Thu  naner  nrecents  the  results  of  research  and  development  work  that  has  focused  on  rapid-prototyping  and  experimenting  with 
advanced  Dressing  methodologies  to  support  C3I  Centers  of  the  future,  particularly  those  that  require  intelligence  data  correlation/ 
daa  fusion  and  operational  use  of  those  data  in  a  timely  manner.  Four  technologies  applicable  to  C3I  Centers  are  addressed  in  this 

paper. 

1  Automatic  Message  Processing  and  Correlation:  Two  algorithmic  correlators,  one  each  for  wmmunications  and  electronics 
d«reSemented  to  create  and  update  an  intellWorder-of-Battle  Data  Base 

statistical  parameter  and  attribute  matching  techniques.  They  function  on  parsed  and  filtered  JINTACCS  messages  and  are 
under  analyst  control,  yet  perform  automatically. 

9  rnnneratine  Exoert  Systems;  Three  cooperating  expert  systems  are  used  to  examine  the  Order-of-Battle  Data  Base  created  by 
the  previously  described  algorithmic  correlators.  These  are  a)  Correlation  and  Association  Enhancement,  b) 

Traokiup  and  c)  Activity  Monitoring.  These  expert  systems  use  knowledge  bases,  stored  in  the  form  of 
semantic  network  of  frames,  in  rules  and  in  computational  functions.  The  knowledge  base  cratains  doctrinal  data,  emitter 
characteristics,  and  analytical  heuristics.  An  endorsement  method  is  used  to  reason  with  uncertainty. 

^  Natural  Language  Text  Line  Processor;  The  natural  language  technique  uses  a  case  frame  grarnmar  to  parse  the  JINTACCS 
mSes  syntaftic  interpret  them  semantically.  This,  processor  provides  recognition  and  expansion  of  acrony^d 

abbrevlatioL,  interpretation  of  free  form  text  and  multiple  text  lines  in  the  same  message,  prompts  the  analyst,  and  formats 
the  results  into  frames  for  use  by  the  expert  systems. 

4  WeaDoneerine/Targeting  Functions;  Algorithms  and  analyst  interfaces  were  developed  and  implenaented  to  provide  automated 
S-weapoS  pt  and  use  of  the  Joint  Munitions  Effectiveness  Manual  mjodology^  These 

Swrittas  o~  automltically  against  the  intelligence  Order-of-Battle  Data  Base  and  perform  the  following  functions: 
calculates  opfiraum  allocation  of  weapons  to  an  array  of  targets,  determines  optimum  allocation  of  loaded  weapons  to  aircraft, 
calculates  single-sortie  probability  of  damage. 

These  technologies  are  integrated  into  a  C3I  Testbed  System  that  is  used  to  demonstrate  and  evaluate  the 

processing  techniques  to  the^  domain  of  tactical  intelligence  processing,  analysis,  and  to  timely  use  of  those  data  by  a  tactical  battle 
manager. 


131 


EXPERIMENTATION  WITH  COOPERAHNQvBXPERT  SYSTEMS  NATURAL 
LANGUAGE  PROCESSING,  AND  ALGORITHMIC  TECHNIQUES  TO  PERFORM  CORRELATION 
AND  DATA  FUSION  IN  A  TACTICAL  ENVIRONMENT 


James  B.  Nelson 


Lockheed  Austin  Division 
O/TO-10,  B/30F 
6800  Burleson  Road 
Austin,  Texas,  78744 
(512)  448-9488 


INTRODUCTION 

Many  existing  tactical  command,  control,  communica¬ 
tions,  and  intelligence  centers  are  severely  limited  in  their 
ability  to  process  automatically  the  large  volume  of  multiple- 
format,  multisource  message  data  they  receive.  Currently 
fielded  systems  lack  the  processing  sophistication  that  is 
needed  to  reduce  this  voluminous  amount  of  data  into  useful, 
manageable  information.  This  shortfall  in  processing  capabil¬ 
ity  of  current  systems  has  stimulated  the  development  of  new, 
improved,  state-of-the-art  processing  techniques  that  are 
more  accurate,  save  time,  and  provide  the  battle  manager  with 
useful,  reliable,  automated  tools. 


Purpose 

The  purpose  of  this  project  was  to  investigate,  develop, 
demonstrate,  and  evaluate  the  application  of  advanced  proc¬ 
essing  techniques  (automatic  message  processing,  algorithmic 
correlation/data  fusion,  natural  language  processing,  expert 
systems,  and  relational  data  base  interfaces)  to  the  domain  of 
tactical  intelligence  processing  and  analysis  and  to  examine 
the  timely  use  of  those  data  by  a  tactical  battle  manager. 


Five  specialized  vignettes  were  created  (by  domain  ex¬ 
perts)  and  added  to  a  1990s  Fulda  Gap  war  scenario.  These 
were  input  to  the  Testbed  system  (1)  to  determine  the  effec¬ 
tiveness  of  the  algorithmic  and  heuristic  approaches  to  corre¬ 
lation/data  fusion,  (2)  to  observe  how  well  the  Natural  Lan¬ 
guage  Text  Line  Processor  (NLTLP)  and  Expert  Systems  (ES) 
performed,  and  (3)  to  ascertain  if  a  dynamically  created  and 
updated  intelligence  Order-of-Battle  (OB)  could  be  used 
automatically  by  a  Weaponeering  (Targeting)  function  of  a 
tactical  C3I  center. 


Vignette  I.  Delineation  of  Unit  Boundaries.  This  vi¬ 
gnette  is  structured  to  capture  the  knowledge  an  analyst  ap¬ 
plies  when  trying  to  determine  the  area  of  interest  of  a  threat 
division,  the  forward  line  of  troops,  the  boundaries  between 
forces  adjacent  to  the  division,  and  the  boundaries  of  subordi¬ 
nate  regiments.  Recognition  precepts  include  inferences  on  air 
defense  assets,  critical  node  location,  and  artillery  command 
and  control. 

Vignette  II.  Recognition  of  Air  Defense  Artillery  Assets. 
This  vignette  focuses  on  tactical  deployment  of  division  air 
defense  assets:  their  electronic  signatures  and  communica¬ 
tions  links.  It  deals  with  the  thought  processes  that  an  analyst 
uses  to  identify  those  assets,  locate  them,  and  track  them. 
Recognition  precepts  include  target  acquisition  battery  infer¬ 
ences,  echelon  and  organizational  inferences,  relative  location 
inferences  for  emitters,  and  overall  deployment  inferences. 

Vignette  HI.  Identification  and  Location  of  Command 
Posts.  This  vignette  is  designed  to  activate  the  thought  process 
of  the  analyst  when  he/she  identifies,  locates,  relates,  and  in¬ 
terprets  the  significance  of  Command  Posts.  Recognition  pre¬ 
cepts  include  signatures,  geolocations,  organizational  data,  in¬ 
teractions,  and  tactical  inferences. 

Vignette  IV.  Recognition  of  Attack  Activities.  This  vi¬ 
gnette  provides  the  stimulus  necessary  to  activate  those  ES 
rules  that  deal  with  recognition  of  the  type  of  combat  activity 
about  to  unfold.  Recognition  precepts  include  communications 
activity,  special  equipment,  frontage  size,  and  asset  deploy¬ 
ment  inferences. 

Vignette  V.  Enhancement  of  Algorithmic  Correlation. 
This  vignette  is  designed  to  stimulate  the  refinement,  assess¬ 
ment,  and  accuracy  of  OB  Database  files  of  emitters  and 
units.  It  permits  the  ES,  which  contains  multiple  hypotheses 
about  emitters  or  units,  to  recorrelate  questionable  decisions 
made  by  the  algorithmic  correlator. 
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The  overall  system  (C3I  Testbed)  is  a  fully  concurrent, 
multiuser  information  processing  system  where  processing  is 
divided  into  functional  blocks  with  well-defined  interfaces  be¬ 
tween  modules.  The  Testbed  functional  architecture  fig¬ 
ure  1)  permits  experimentation  and  demonstration  of  inte¬ 
grated  technologies  as  applied  to  the  intelligence  and 
weaponeering  functions  of  a  tactical  C3I  center. 

Fully  formatted  JINTACCS  messages  are  input  to  the 
Testbed  by  an  AirLand  Battle  simulator,  which  uses  a  1990s 
Fulda  Gap  war  scenario  (enemy  events  and  OB)  and  a  variety 
of  airborne  sensor  models.  The  Testbed  automatically  parses 
(decodes)  and  filters  the  JINTACCS  messages  according  to 
tables  that  are  set  and  easily  changed  by  the  Intelligence  Ana¬ 
lyst.  Two  algorithmic  correlators  (statistical  and  figure-of- 
merit  based)  perform  automatically  to  read  the  message  data 
and  to  create  and  update  emitter  files  within  an  OB  Database. 

A  Data  Fusion  function  examines  the  emitter  files  cre¬ 
ated  by  the  correlators,  adds  the  information  produced  by  the 
artificial  intelligence  (AI)  techniques,  then  combines  the  data 
to  form  units  and  weapons  systems  within  the  OB  that  repre¬ 
sents  the  most  complete  information  about  entities  on  the  bat¬ 
tlefield. 


The  AI  techniques  include  the  NLTLP  and  distributed 
cooperating  ES.  The  NLTLP  reads  and  interprets  the  English 
language  text  lines  of  the  JINTACCS  messages,  reformats  the 
data,  and  passes  the  results  to  the  ES.  The  ES  examines  the 
OB  Database  and  the  NLTLP  output  to  determine  the  pres¬ 
ence  of  certain  types  of  units  on  the  battlefield,  their  geo¬ 
graphic  operating  boundaries,  and  activity  state.  Additionally, 
the  ES  recognizes  and  corrects  certain  types  of  errors  not  han¬ 
dled  by  the  algorithmic  correlators. 

The  Weaponeering/Targeting  function  automatically  ac¬ 
cesses  the  intelligence  OB  Database,  compiles  target  lists,  and 
calculates  target-weapon  pairing  solutions  based  on  a  global 
optimization  algorithm. 

An  imagery  processing  capability  is  in  the  planning 
stages  at  this  time. 

An  Air  Task  Order  (ATO)  generation  capability  is  in 
the  development  stage. 

The  system  architecture  permits  a  wide  range  of  experi¬ 
mentation  possibilities  for  the  testing  of  specific  modules,  al¬ 
gorithms,  or  fully  integrated  components. 
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Figure  2.  Hardware/Software  Modules  and  Interfaces 


Hardware/Software  Imolementattnn 

The  approach  used  to  implement  the  C3I  Testbed  Sys¬ 
tem  is  characterized  as  a  balanced  integration  of  conventional 
processing  technology  with  AI  processing  techniques  in  a  dis¬ 
tributed  environment.  Each  technology  is  applied  to  the  intel¬ 
ligence  analysis  and  targeting  problems  in  the  manner  deemed 
most^  appropriate.  This  approach  does  not  attempt  a  total  AI 
solution  nor  does  it  attempt  to  force  AI  techniques  on  aspects 
of  the  problem  that  have  already  been  solved  by  algorithmic 
methods.  Figure  2  shows  the  hardware  connectivity  and  the 
software  major  module  integration  of  conventional  algorithmic 
processes  with  AI  heuristic  processes. 

Distributed  Processing  Interface.^ 

This  project  emphasized  the  use  of  distributed  process¬ 
ing  techniques  as  a  way  to  achieve  fast  and  useful  interfaces 
between  the  various  workstations  and  processes.  These  inter¬ 
faces  permit  the  ES,  NLTLP,  algorithmic  correlator  analyst 
displays,  Weaponeer,  and  data  base  management  functions  all 
to  reside  on  separate  processors,  yet  be  accessible  by  any. 
Two  primary  classes  of  interfaces  are  used:  RS232  software 
interface  and  TCP/IP  software.  In  addition  to  vendor-devel¬ 
oped  TCP/IP  system  software  for  specific  machines,  we  devel¬ 
oped  a  remote  procedure-call  applications  layer  for  interface 
between  the  Apollo  and  Symbolics  machines. 


Software  Tools  and  Programming  Languages 

The  Testbed  System  uses  a  variety  of  software  tech¬ 
niques  to  niake  the  system  work.  These  are  summarized  as 
follows: 

Simulator:  Ada,  Vax  11/780 

Expert  Systems:  Common  Lisp,  Automatic  Reasoning 
Tool  (ART)  ES  Shell,  Symbolics.  ART  was  chosen  for  its 
graphics,  rule  definition  and  compilation,  confidence  levels, 
and  viewpoints  (blackboard)  capabilities. 

Natural  Language:  Common  Lisp,  Language  Craft 
NLP  Shell,  VAX,  and  Symbolics.  Language  Craft  was  chosen 
because  it  is  a  general-purpose,  natural-language  processing 
tool,  it  permits  the  developer  to  concentrate  on  vocabulary 
and  semantics  rather  than  mechanics  of  parsing  and  semantic 
interpretation,  it  has  a  well-defined  interface  to  Common 
Lisp,  and  runs  on  both  VAX  and  Symbolics. 

Map  Graphics:  C,  Apollo  (also  runs  on  Silicon  Graph¬ 
ics,  Sun,  MicroVAX).  Digital  map  graphics  display  technology 
(SoftCopy  Map  Display  System)  contains  DMA  data  (DTED, 
DFAD)  and  World  Data  Base  II  and  has  user  friendly  software 
that  permits  extensive  map  manipulation  and  control. 

User  Interface  Manager:  C,  Apollo  (also  runs  on  Sili¬ 
con  Graphics,  Sun,  MicroVAX).  System  permits  creation/ 
change  of  displays  in  minutes  by  a  nonprogrammer. 

Correlators:  FORTRAN,  Apollo. 

Data  Base:  IDL  and  Progress  Relational  DB  Access, 
Britton-Lee. 

Message  Processing:  FORTRAN,  Apollo. 
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AmoRnmur.  correlators 

Both  the  Tactical  Electronics  Intelligence  (TACELINT) 
and  the  Tactical  Report  (TACREP)  correlation  algorithms 
function  automatically  within  the  Testbed  message— handling 
architecture.  Each  has  a  set  of  specialized  interactive  tools 
that  permit  an  analyst  to  control  automatic  correlation  vari¬ 
ables:  review,  edit,  or  change  the  results  of  automatic  correla¬ 
tion  and  make  manual  correlations. 

TAr.RT.TNT  CORRELATOR 

The  TACELINT  Correlator  performs  the  following  pri¬ 
mary  functions: 

•  Attempts  to  correlate  signal-of-interest  messages  to  exist¬ 
ing  emitter  OB  Data  Base  files  automatically 

•  Updates  emitter  OB  Data  Base  files  automatically 

•  Sends  graphic  updates  to  the  map  graphics  display  (ana¬ 
lyst  workstation). 

The  TACEUNT  algorithm  is  based  on  a  scoring  and 
weighting  system  that  calculates  the  statistical  differences  be¬ 
tween  the  emitter  parameters  contained  in  a  message  and  the 
parameters  of  emitters  held  in  the  data  base  that  fall  within  a 
tolerance  gate  set  by  the  intelligence  analyst.  Those  OB  Data 
Base  emitters  that  fall  within  the  tolerance  gate  are  considered 
to  be  reasonable  candidates  for  correlation.  Each  parameter 
of  each  emitter  is  scored  and  weighted  (by  any  criteria  as 
determined  by  the  analyst,  e.g.,  sensor  measurement,  time  de¬ 
lay,  etc.).  Next,  each  candidate  is  tested  for  geopositional  fea¬ 
sibility  and  scored.  Total  scores  are  then  calculated  for  each 
candidate  versus  the  message  emitter.  Finally,  the  algorism 
compares  each  candidate.  If  a  single  best  candidate  (heuristi- 
cally  determined  by  the  analyst  and  input  to  the  algorithm)  is 
identified,  a  correlation  is  made.  Otherwise,  candidates  are 
ranked  and  filed  in  the  OB  Data  Base  as  potential  (ambigu¬ 
ous)  correlations.  Each  ambiguous  emitter  is  reconsidered  as 
new  messages  are  input  to  the  TACELINT  correlator. 


TACREP  CORRELATOR 

The  TACREP  correlator  uses  parametric  association 
and  attribute-matching  techniques  to  correlate  message  data 
with  emitters  held  in  the  OB  Data  Base.  Automatic  correla¬ 
tion  occurs  when  certain  combinations  of  data  elements  in  the 
message  match  exactly  or  within  tolerances  with  the  data  ele¬ 
ments  contained  in  the  data  base.  If  a  unique  identifier  is  pre¬ 
sent  in  both  the  message  and  the  OB  Data  Base,  if  the  candi¬ 
date  meets  a  geopositional  feasibility  test,  and  if  the  candidate 
meets  a  scoring  threshold  (set  by  the  analyst),  a  correlation  is 
made.  Those  attempts  at  correlation  that  do  not  meet  the 
minimum  score  result  in  the  message  data  being  returned  to  a 
message  file  for  future  attempts  at  correlation.  The  TACREP 
correlator  does  not  create  ambiguities. 


DATA  FUSION 

The  Data  Fusion  function  uses  the  emitter  OB  Data 
Base  files,  previously  created  by  the  TACELINT  and  TACR^ 
correlators,  to  create  and  maintain  weapons  systems  and  units 
in  the  OB.  It  also  uses  the  data  provided  by  the  ES  and  the 
analyst  workstation. 

Data  Fusion  performs  automatically,  making  inferences 
about  the  battlefield.  At  the  first  level,  inferences  proceed  di¬ 
rectly  from  a  single  fact,  e.g.,  the  existence  of  a  weapons  sys¬ 
tem  can  be  inferred  directly  from  the  existence  of  certain  . 
emitters.  Some  unit  types  can  also  be  inferred  in  this  manner. 
A  second  level  of  inferences  occur  when  inferences  are  made 
based  on  several  facts  that  do  not  imply  anything  separately, 
but  when  considered  together  imply  something  about  the  OB. 
For  example,  the  existence  of  several  different  entities,  lo¬ 
cated  within  a  certain  distance  of  each  other,  or  the  determi¬ 
nation  of  a  parent-child  relationship  between  two  or  more  en¬ 
tities,  can  imply  much  more  than  the  existence  of  a  single 
entity. 

The  Data  Fusion  function  performs  the  following  func¬ 
tions: 

•  Groups  entities  into  systems 

•  Determines  battlefield  positions  for  systems  based  on  lo¬ 
cations  of  component  parts 

•  Deduces  the  presence  of  units  from  the  deployment  of 
specialized  equipment 

•  Assigns  systems  and  equipment  to  the  controlling  unit 

•  Determines  battlefield  unit  positions 

•  Determines  unit  organizational  hierarchy. 

The  Testbed  System  uses  tightly  coupled  OB  Data  Base, 
Correlation,  and  Data  Fusion  functional  modules  as  a  com¬ 
plete  OB  management  system.  Each  of  the  automatic  proc¬ 
esses  is  responsible  for  maintaining  its  specialty  data  in  the 
central  data  base  and  on  the  map  display.  All  changes  to  the 
OB,  whether  initiated  by  the  analyst,  ES,  Data  Fusion,  or  Cor¬ 
relator,  maintain  the  integrity  of  the  OB  Data  Base.  For  exam¬ 
ple,  the  parametric  details  of  TACELINT  correlation  may  be 
unknown  by  Data  Fusion,  but  if  the  emitter  had  moved,  the 
communication  of  that  fact  to  Data  Fusion  would  cause  the 
Data  Fusion  algorithm  to  recalculate  the  position  for  the  unit 
containing  that  emitter. 
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Figure  3.  AI  Techniques  Integrated  with  Testbed  System 


ARTTFICIAL  INTELLIGENCE  TECHNTOTJF.S 

Seven  component  subsystems  using  AI  techniques  were 
integrated  into  the  Testbed  System.  These  components  are  or¬ 
ganized  as  illustrated  in  Figure  3.  The  ES  includes  three  coop¬ 
erating  subexperts  and  two  global  support  functions. 

Correlation  and  Association  Enhancement 

The  Correlation  and  Association  Enhancement  (CAE) 
subexpert  looks  for  constraint  violations  in  the  output  of  the 
algorithmic  correlators,  directs  the  correlators  to  recorrelate 
the  associated  messages,  and  communicates  with  the  cor¬ 
relator  in  a  manner  functionally  similar  to  that  of  the  human 
analyst.  The  CAE  works  in  conjunction  with  the  FRT  to  per- 
forrn  constraint-based  correlation  and  data  fusion.  Con¬ 
straint-based  correlation  is  performed  using  constraints  along 
one  or  more  of  several  different  problem  space  dimensions 
(or  classes):  Parametric,  Geopositional,  Doctrinal,  Terrain, 
Activity,  and  Multisource.  CAE  is  closely  coupled  with  the 
algorithmic  correlator  and  has  a  feedback  loop  with  it  to  pass 
information  on  recorrelation  and  change  of  OB  Data  Base. 

Force  Structure  Recognition  and  Tracking 

The  Force  Structure  Recognition  and  Tracking  (FRT) 
subexpert  accomplishes  the  following:  hypothesizes  force 
structures  from  the  association  of  emitter  reports;  determines 
the  types  of  units  represented  using  doctrinal,  table  of  organi¬ 
zation  and  equipment  (TO&E),  and  analytical  knowledge; 
tracks  the  movement  and  confidence  associated  with  the  force 
struchjres  and  units.  The  FRT  formulates  and  compares  the 
positional  clusters  formed  by  individual  emitters,  determines 
subclusters,  and  examines  the  relationships  between  clusters 
to  form  an  opinion  of  the  existence  of  certain  types  of  units. 
This  subexpert  also  compares  the  total  OB  known  to  exist 
within  an  area  of  interest  to  doctrinal  data  to  determine  if 
there  is  agreement  between  the  perceived  and  doctrinal  OB.  If 
discrepancies  are  found,  they  are  reported  to  the  analyst.  In 
addition,  the  FRT  performs  specific  unit  identification,  includ¬ 
ing  echelon  and  subordination. 


Truth  Maintenance 

The  Truth  Maintenance  Facility  maintains  the  interrela¬ 
tionships  between  objects  and  permits  the  system  to  backtrack 
whenever  a  given  object  characteristic  or  occurrence  is  found 
to  be  in  error  or  is  contradicted.  These  interrelationships  are 
derived  from  the  endorsements  of  each  object  maintained  for 
reasoning  under  uncertainty  and  allow  for  chaining  from  one 
object  and  fact  dependency  to  another. 

Explanation  Facility 

The  Explanation  Facility  provides  an  English-like  ex¬ 
planation  of  the  reasoning  process  behind  each  decision  made 
by  the  three  subexperts.  This  facility  responds  to  analyst  re¬ 
quests  for  explanation  regarding  any  emitter,  unit,  or  activity 
indicator.  The  explanations  are  derived  from  the  endorse¬ 
ments  of  each  object  maintained  for  reasoning  under  uncer¬ 
tainty  and  include  an  audit  trail  of  the  inferences  performed 
rather  than  just  a  list  of  rule  names.  Each  rule  included  Within 
any  of  the  ES  subexperts  is  coded  with  a  brief  Engijsh-lan- 
guage  explanation  of  the  rule  function,  and  when  a  riile  fires, 
an  endorsement  is  created  for  the  object  of  interest  and  is 
stored  with  the  object.  By  using  the  accumulated  endorse¬ 
ments  as  the  basis  for  explanations,  a  separate  audit  trail  does 
not  have  to  be  maintained  and  overall  system  consistency  is 
ensured. 


Activity  Monitoring 

The  Activity  Monitoring  (AM)  subexpert  recognizes  ac¬ 
tivity  indicators  associated  with  the  units  in  the  OB,  accumu¬ 
lates  warning  signs,  and  posts  alerts  to  the  analyst  for  indica¬ 
tors  of  attack.  It  uses  the  output  of  the  Natural  Language 
^ocessor  and  the  output  of  the  algorithmic  correlators.  Activ¬ 
ity  indicators  are  accumulated  by  activity  category  and  by  unit 
type  involved.  The  AM  uses  a  hierarchical  structure  of  indica¬ 
tors,  warnings,  and  alerts  so  that  any  type  of  activity  can  be 
included.  If  multiple  indicators  occur  for  a  given  activity  or 
unit  type,  a  warning  is  provided  to  the  analyst.  If  multiple 
warning  are  created  regarding  a  particular  activity  type,  an 
alert  is  created  and  issued  to  the  analyst. 
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Figure  4.  Natural  Language  Functions  Integrated  with  testbed  System 


Natural  Language  Text  Line  Processor 

The  NLTLP  was  designed  to  combat  the  difficulties  of 
interpreting  free  text  and  creating  a  usable  form  of  data  for 
the  ES.  The  most  important  goals  were  to  examine  the  tech¬ 
niques  needed  for  processing  free  text  and  to  create  an  auto¬ 
matic  process  to  accomplish  this.  The  techniques  explored 
were  the  caseframe  grammar  used,  along  with  discourse 
analysis  techniques  such  as  using  a  context  mechanism,  to 
complete  missing  values.  These  techniques  were  written  in 
Common  Lisp,  along  with  the  grammar  written  for  use  with 
the  Language  Craft  parser.  The  techniques  were  initially  im¬ 
plemented  as  a  automatic  process;  however,  an  entirely  auto¬ 
matic  process  does  not  appear  to  be  the  best  solution  because 
ambiguous  (multiple)  parses  can  occur  unless  the  grammar  is 
constructed  very  carefully.  Ambiguous  parses  can  be  pre¬ 
sented  to  the  analyst  to  resolve,  saving  computer  time.  The 
integration  of  the  NLTLP  within  the  Testbed  System  is  shown 
in  Figure  4. 

NTTLP  Functions  and  Design 

The  Natural  Language  Text  Line  Processor  accepts  the 
fixed  fields  and  free-form  text  lines  of  both  TACEUNT  and 
TACREP  messages,  parses  the  free-form  text,  and  interprets 
the  meaning  of  this  text  in  the  context  of  the  fixed  fields  and 
other  text  lines.  The  results  of  this  processing  are  records  that 
can  be  processed  and  interpreted  by  the  ES.  The  ^TLP  in¬ 
cludes  preprocessing  functions  to  handle  abbreviations,  acro¬ 
nyms,  and  simple  misspellings.  It  handles  a  variety  of  activity 
types,  verbs,  nouns,  and  modifiers.  It  can  distinguish  tenses 
and  can  correctly  interpret  time  and  place  information.  The 
NLTLP  uses  a  caseframe  grammar  approach  to  parsing,  and 
semantic  interpretation  makes  the  NLTLP  easy  to  extend  to 
new  reports,  activities,  objects,  and  modifiers. 


Preprocessing  of  Free  Text 

Preprocessing  is  done  on  text  lines  before  they  are 
parsed  by  grammar.  Several  preprocessing  steps  are  used; 

•  Message  handling  with  and  without  text 

•  Breakdown  of  multiple  text  lines  into  individual  text  lines 

•  Preparation  of  each  text  line  (acronym  expansion,  separa¬ 
tion  of  numbers  and  characters  in  time  and  location 
fields,  addition  of  case  markers  to  enhance  parsing,  and 
handling  of  abbreviations  not  allowed  by  Language  Craft. 

Components  of  Grammar 

The  grammar  used  for  the  NLTLP  processing  is  the 
caseframe  grammar  that  is  used  with  the  Language  Craft  par¬ 
ser  (Plume),'  consisting  of  the  four  main  components: 
caseframes,  rewrite  rules,  irregular  morphology  rules,  and  ab¬ 
breviation  rules. 

Postprocessing  Rule 

Postprocessing  rules  operate  on  the  instantiated 
caseframe  generated  by  the  parsing  of  a  text  line.  Three 
forms  of  postprocessing  rules  occur: 

•  Mapper  Rules.  These  are  used  to  transform  the  caseframe 
instances  output  by  the  parser  into  caseframes  for  input  to 
the  USPifier  rules. 

•  USPifier  Rules.  These  are  used  to  convert  mapper  output 
into  USP  forms  suitable  for  a  particular  application. 

•  Context  Resolution  Rules.  A  context  schema  is  created 
from  analyzing  each  text  line,  and  if  a  resultant  schema  is 
missing  information,  previous  schemas  are  checked  to  fill 
the  missing  values, 

Figure  5  provides  an  overview  of  the  NLTLP  functional 

flow. 
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Figure  5.  Natural  Language  Text  Line  Processor  Functional  Modules 


AI  Techniques  Control  Mechanisms 

To  achieve  a  balance  between  control  (reduce  the  time 
spent  responding  to  a  particular  input)  and  flexibility  (sponta¬ 
neous  handling  of  expected  and  unexpected  inputs),  the  ES 
design  consists  of  the  following:  cooperating  subexperts,  use 
of  an  endorsement  method  for  reasoning  under  uncertainty, 
and  a  mixed-initiative  user  interaction. 

•  Cooperating  Subexperts:  Primary  control  was  established 
in  the  FRT  subexpert  because  algorithmic  correlators 
have  already  processed  much  of  the  data.  However,  the 
system  uses  trigger  events  to  assist  in  transitions  between 
subexperts  so  that  certain  events  cause  primary  control  to 
switch  to  the  appropriate  subexpert. 

•  Endorsement  Method:  This  method  captures  endorse¬ 
ments  or  votes  for  or  against  a  particular  conclusion, 
which  is  then  reviewed,  summarized,  and  assigned  a  con¬ 
fidence  level.  As  votes  are  collected,  a  record  of  the  sup¬ 
port  for  the  typing  of  each  object  is  kept,  and  the  evidence 
is  increased  or  decreased  as  appropriate.  Endorsements 
also  serve  as  a  basis  for  constructing  explanations  of  sys¬ 
tem  decisions  and  for  truth  maintenance. 

•  Mixed-Initiative  User  I/A:  This  approach  to  ES  control 
allows  both  the  analyst  and  the  ES  to  direct  the  flow  of 
processing. 


Relational  Data  Base  Interface 

Relational  data  bases  provide  an  especially  flexible  or¬ 
ganization  system  for  storing  large  volumes  of  data  and  data 
relationships  in  tabular  form.  Interfaces  between  relational 
data  base  management  systems  and  AI  systems,  like  the 
NLTLP  and  ES,  permit  the  latter  to  retrieve  data  and  informa¬ 
tion  automatically  and  selectively  and  store  the  results  of  the 
processing  for  later  use  by  both  systems  and  operators. 

The  relational  data  base  interface  allowed  the  NLTLP 
and  ES  to  share  a  common  relational  data  base  for  the  OB 
with  the  algorithmic  correlators  and  the  analyst  interface.  It 
allows  the  ES  to  submit  data  base  queries  autonomously  and 
update  requests  to  the  management  system. 

Analyst-Machine  Interface 

The  project  demonstrated  the  use  and  integration  of 
both  AI  and  conventional  technologies  for  the  analyst-ma¬ 
chine  interface.  The  interface  includes  a  menu-based  query- 
and-command,  interactive  map  graphics,  and  an  ES  monitor- 
ing-and-control  function.  For  the  ES,  the  analyst  can  ask  for 
descriptions  of  objects  in  the  OB  and  explanations  for  their 
existence.  The  correlation  analyst  can  query,  move,  change, 
and  ask  for  unit  hierarchy  or  parametric  data,  uncertainty  el¬ 
lipses,  unit  or  emitter  histories  as  well  as  perform  other  ana¬ 
lytic  functions. 
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^^^APONEERTNG/TARGETING  FUNCTIONS 

The  Testbed  System  intelligence  analysis  functions  were 
integrated  with  a  battle  management  function:  targeting  and 
weaponeering.  This  interface  permits  the  battle  management 
functions  to  access  a  dynamically  created  and  updated  intelli¬ 
gence  OB,  as  it  is  happening.  During  system  tests,  we  were 
able  to  observe  the  input  of  message-containing  emitter  data 
from  the  simulator  into  the  Testbed  Parser  and  Filter,  Cor¬ 
relator,  Data  Fusion,  through  the  NLTLP  and  ES,  into  the  OB 
Data  Base,  and  finally,  watch  the  data  appear  as  a  target  at 
the  weaponeering/targeting  analyst  display. 

Target-Weapon  Pairing 

The  Target- Weapon  Pairing  (T-WP)  algorithms  deter¬ 
mine  the  optimum  allocation  of  weapons  systems  on  alert  (or 
with  short  response  times)  to  an  array  of  targets  that  have 
been  selected.  The  allocation  is  based  on  the  OB,  weapons 
information  from  an  Assets  Data  Base,  the  Single-Sortie 
Probability  of  Damage  from  the  Joint  Munitions  Effectiveness 
Manual  (JMEM),  and  the  analyst’s  input  of  weights  to  be 
given  for  such  factors  as  attrition,  weapon  system  cost,  and 
scarcity.  The  T-WP  algorithms  pair  available  weapon  re¬ 
sources  to  a  list  of  prioritized  targets  based  on  a  criteria  func¬ 
tion,  the  number  of  weapons  available,  and  the  number  of 
weapons  needed  to  attack  at  the  desired  level  of  effectiveness. 
The  weapons/targeteer  analyst  can  chose  targets  on  the  OB 
map  display  using  the  cursor  or  by  inputting  target  identifiers. 
Solutions  are  automatically  generated  and  displayed.  They 
may  be  saved  while  new  criteria  are  entered  for  a  different 
solution,  compared,  and  either  rejected  or  accepted.  The  algo¬ 
rithm  handles  up  to  25  targets  simultaneously  to  pair  up  with 
12  weapons  systems  standard  conventional  loads. 

Target-Weapon  Scheduling 

The  scheduling  algorithm  determines  the  best  way  to 
load  and  assign  weapon  systems  to  all  targets  in  a  given  Air 
Task  Order  (ATO)  generation  cycle.  It  compares  available 
weapon  assets  with  a  target  list  based  on  multiple  sorties  per 
weapon  and  the  cost  effectiveness  of  all  target-weapon  pairs. 


The  algorithm  maximizes  the  sum  of  the  cost-effectiveness 
values  for  the  target-weapon  pairs  subject  to  the  following 
constraints:  number  of  available  weapons,  the  minimum  time 
from  take-off  for  an  aircraft  from  a  given  weapon  set,  num¬ 
ber  of  sorties  an  aircraft  can  accomplish  in  one  day  and  its 
cycle  time,  windows  for  each  target,  and  weather  for  both  the 
aircraft  home  base  and  the  target.  The  approach  avoids  the 
trap  of  assigning  the  best  weapons  to  the  highest  priority  tar¬ 
gets  by  detecting  anomalies  or  special  cases  in  the  total  tar¬ 
get-weapon-time  matrix  for  the  scheduling  period. 

.Toint  Munitions  Effectiveness  Manual 

JMEM  methodology,  along  with  user  functions  that  per¬ 
mit  problem  specification  and  standalone  execution  of  the  al¬ 
gorithm,  were  implemented  in  the  system.  The  JMEM  algo¬ 
rithm  calculates  the  weapon-effectiveness  estimates  of  the 
single— sortie  probability  of  damage  and  number  of  weapons 
needed  to  achieve  the  probability  of  damage  for  a  given  target 
type  and  weapon  combination. 

Target-Weapon  Pairing  System:  Aided  Decisions 

Under  development  within  the  Battle  Management  labo¬ 
ratory  is  a  rapid  prototyping  and  evaluation  of  decision-aids 
that  would  further  automate  the  battle  management  functions 
of  tactical  C3I  centers  of  the  future.  Figure  6  illustrates  a  part 
of  the  decision-aiding  necessary  to  advance  the  automation  of 
this  cycle. 

Many  existing  tactical  command,  control,  communica¬ 
tions,  and  intelligence  centers  are  severely  limited  in  their 
ability  to  process  the  large  volume  of  multiple-format,  multi¬ 
source  message  data  automatically.  Currently  fielded  systems 
lack  the  processing  sophistication  that  is  needed  to  reduce  this 
great  quantity  of  data  into  useful,  manageable  information. 
This  shortfall  in  processing  capability  of  current  systems  has 
stimulated  the  development  of  new,  improved,  state-of-the- 
art  processing  techniques  that  are  more  accurate,  save  time, 
and  provide  the  battle  manager  with  useful,  reliable,  auto¬ 
mated  tools. 
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RESULTS  AND  CONCLUSIONS 

The  most  significant  operational  conclusion  regarding 
the  application  of  artificial  intelligence  technologies  to  tactical 
intelligence  processing  and  analysis  is  that  these  technologies, 
including  the  ES  and  NLTLP,  can  significantly  enhance  the 
support  provided  to  the  analyst  over  conventional  methods  if 
they  are  applied  to  the  more  abstract  and  less  quantitative 
portions  of  the  problem  domain.  These  capabilities  comple¬ 
ment  proven  conventional  and  algorithmic  approaches  and 
provide  an  intelligent  assistant  for  the  analyst  in  the  areas  of 
correlation  enhancement,  text  processing,  and  intelligent  data 
base  interface.  The  NLTLP  and  ES  provide  a  faster  method  of 
handling  high  volumes  of  messages  because  they  can  reason 
about  incomplete  and  uncertain  data,  provide  inferences  at 
multiple  levels,  and  maintain  multiple  contexts. 

The  Testbed  System  demonstrated  that  analyst  expertise 
can  be  embedded  in  a  system.  The  ES  knowledge  base  in¬ 
cludes  the  doctrinal,  TO&E,  parameter  data,  and  analytical 
expertise  used  by  the  analyst  to  process  tactical  intelligence 
reports.  The  ES  inference  engine  uses  this  knowledge  to  per¬ 
form  analyses,  reason  about  uncertain  data,  and  present  (dis¬ 
play)  the  current  tactical  situation.  The  NLTLP  knowledge 
base  includes  the  grammar,  vocabulary,  and  contextual  asso¬ 
ciations  that  must  be  understood  by  an  analyst  to  interpret 
these  reports. 

The  use  of  a  relational  data  base  interface  significantly 
reduced  the  analyst  workload  by  automatically  retrieving  and 
updating  data  and  information  from  data  bases  for  use  by  the 
ES  and  NLTLP. 

Natural  language  processing  can  extract  useful  informa¬ 
tion  from  free-form  text,  associate  this  information  with  the 
formatted  data  (report  context),  and  format  it  for  use  by  the 
ES.  The  NLTLP  processes  messages  much  more  rapidly  and 
accurately  than  human  analysts.  The  use  of  caseframes  to 
support  natural  language  processing  permits  smooth  integra¬ 
tion  of  an  NLTLP  and  ES. 

Advanced  and  conventional  techniques  can  be  inter¬ 
faced,  can  use  a  common  analyst-machine  interface,  and  can 
be  implemented  in  a  distributed  environment  that  shares  a 
common  data  base. 

The  development  of  customized  knowledge  bases,  ex¬ 
pert  systems,  natural  language  processors,  and  interfaces  for 


each  user  or  for  each  tactical  unit  is  not  necessary.  A  generic 
model  of  knowledge  and  metaknowledge  can  be  developed 
that  will  serve  as  a  basis  for  supporting  the  majority  of  tactical 
analysis  needs. 

Most  specific  knowledge  in  a  knowledge  base  can  be 
stored  in  frame  structures  and  general  rules  rather  than  using 
specific  rules.  The  frame,  a  flexible  record-like  structure  with 
slots  (fields),  relationships,  and  attached  procedures,  can  be 
used  to  represent  most  of  the  specific  knowledge  in  a  knowl¬ 
edge  base,  and  general  rules  can  be  coded  that  reference  en¬ 
tire  classes  of  patterns  rather  than  very  specific  patterns  and 
data  values. 

A  single  deterministic  method  (endorsement)  can  be 
used  as  the  basis  for  reasoning  under  uncertainty,  truth  main¬ 
tenance,  and  explanation. 

The  Testbed  System  demonstrates  a  prototype  of  inter¬ 
operability  and  integration  of  intelligence  analysis  (prepara¬ 
tion  of  the  OB)  and  weaponeering  (targeting)  functions  of  a 
tactical  C3I  center.  This  prototype  indicates  a  significant  time 
savings  in  generating  targeting  solutions  through  the  use  of  an 
automatic  optimization  scheme  and  the  use  of  direct  interface 
with  the  intelligence  OB  Database. 

FUTURE  DIRECTIONS 

Future  research  issues  concerning  the  use  of  advanced 
processing  techniques  for  tactical  C3I  centers  can  be  outlined 
as  follows: 

•  Automated  knowledge  acquisition  and  knowledge  base  ex¬ 
tensibility,  particularly  for  field  use 

•  Cooperating  distributed  ES 

•  Extended  natural  language  processing  with  human  inter¬ 
faces  and  contextual  explanation  generation 

•  Sensor-cueing  based  on  situation  assessment  stemming 
from  ES  such  as  the  Activity  Monitoring  and  Force  Recog¬ 
nition  and  Tracking 

•  Terrain-based  geofeasibility  extension  to  enhance  correla¬ 
tion  accuracy 

•  Extended  automation  and  netting  of  Battle  Management 
functions  with  intelligence  functions. 
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DATA  FUSION  AS  THE  GUIDING  PRINCIPLE  IN  THE  BROAD  AREA  SEARCH  PROBLEM (U) 


Dr.  R.H.  Gofer 


Government  Information  Systems  Division 
Harris  Corporation 
P.O.  Box  98000 
Melbourne,  FL  32902 


(U)  The  problem  of  keeping  track  of  movable  targets  operating  over  broad  areas  of  terrain 
poses  a  significant  challenge  to  the  intelligence  community.  Intelligence  data  in  a  variety  of 
forms  from  a  variety  of  sensor  platforms  is  available,  but  the  time  interval  between  observations 
of  targets  is  likely  to  be  such  that  the  target  positions  will  change;  perhaps  significantly, 
during  the  intervals.  Sensor  observations  of  the  target  area  can  provide  at  least  partial 
information  about  the  operational  status  of  the  target.  This  information  should  be  fused  with 
ancillary  information  about  road  networks,  terrain  features,  weather  factors,  deployment  tactics, 
and  constraints  on  physical  movement.  In  addition  there  is  a  need  for  integration  of  the  sensor 
tasking  processes  to  insure  the  best  possible  surveillance  of  the  broad  areas. 

(U)  This  paper  describes  an  expandable  framework  whereby  an  automated  fusion  tool  can  be 
constructed  to  assist  in  the  solution  of  the  broad  area  search  problem.  The  framework  is 
analogous  in  some  ways  to  the  Kalman-filter  approach  to  state  estimation  problems:  There  is  a 
predictive  component  that  uses  all  available  information  to  produce  continuous-time  estimates  of 
the  locations  of  targets,  and  there  is  an  observation  component  that  provides  new  information  at 
non-uniform  discrete  intervals  of  time.  These  two  kinds  of  information  are  fused  to  produce  an 
improved  estimate  of  the  locations  of  targets,  and  to  guide  future  observation  times  and  places. 
This  is  done  by  an  algorithmic  inference  engine  which  propagates  the  estimate  of  the  state  of  the 
system  (the  target  location)  in  a  way  that  keeps  track  of  the  multiple  possibilities  of  the  new 
unobserved  location  of  targets  that  moved.  Probabilistic  confidence  levels  are  associated  with 
each  possible  predicted  location,  and  sensor  tasking  management  is  then  based  on  maximizing  the 
chance  of  finding  a  moved  target.  The  propagation  process  is  computationally  intensive  but  well 
within  the  state  of  the  art.  Simulation  photographs  of  the  process  taken  during  a  scenario  are 
provided  along  with  a  description  of  the  underlying  framework. 
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DATA  FUSION 

AS  THE  GUIDING  PRINCIPLE  IN  THE  BROAD  AREA  SEARCH  PROBLEM  <U) 


Dr.  R-  H.  Cofer 


Government  Information  Systems  Division 
Harris  Corporation 
P.  0.  Box  98000 
Melbourne,  FL  32902 


<U)  INTRODUCTION 

<U)  Figure  1  shows  a  typical  intelligence 
process  reduced  to  fundamentals.  A  movable 
target  is  acting  out  its  own  scenario 
oblivious  of  any  specific  sensor  tasking  or 
observation.  It  however  generally  realizes 
the  potential  for  observation  and  divides 
its  time  between  deployment  and  broad  area 
relocation  to  avoid  detection. 


UNCLASSIFIED 


Figure  1.  <U)  The  Intelligence  Processing 

Cycle 

(U)  The  intelligence  process  would  like  to 
maintain  contact  of  the  target  but  is 
hampered  by  the  lack  of  continuous 
observation.  The  obvious  question  is  where 
to  place  the  next  sensor  collection  to 
maximize  the  chance  of  picking  up  the 
evading  target  again.  This  leads  naturally 
to  the  problem  of  fusion  of  observation  and 
target  prediction  functions  in  order  to 
properly  task  available  sensor  resources. 

CU)  Solution  requires  due  attention  to 
proper  use  of  disparate  informations: 
terrain  mobility,  target  maneuverability  and 


doctrine,  opportunistic  intelligence 
reports,  and  sensor  availability:  all  as 
discontinuous  functions  of  time  and  space. 
The  key  to  the  solution  is  maintainance  of  a 
Kalman-f i Iter  analogous  set  of  target  state 
probabi 1 ities. 

<U)  The  following  three  sections  discuss 
the  algorithmic  inference  engine  concepts 
used.  Figure  2.  Then  two  overview  examples 
are  provided  to  point  out  features  of  its 
use.  The  principal  conclusion  to  be  drawn 
is  that  the  techniques  proposed  can  provide 
a  significant  aid  to  the  overall 
intelligence  process. 

<U>  TERRAIN  INTERACTIONS 

<U)  During  lack  of  incoming  intelligence, 
the  only  information  available  is  that  of 
terrain  influences  on  target  maneuverabi 1 i ty 
and  velocity-  To  simplify  the  computer 
solution,  terrain  locations  are  expressed  as 
points,  Ci,j),  on  a  square  digitization 
grid. 


<U)  Maneuverabi 1 i ty 

<U)  The  terrain  holds  great  influence  over 
the  target's  maneuvarabi 1 i ty  and  deployment 
patterns.  This  influence  can  be  expressed 
though  use  of  the  following  variables: 

”  Probability  that  a  target 
moving  to  <i,j)  will  stop 

Pjjj^jQ^iFj)  =  Probability  that  a  target 
stopped  at  (i,J>  will 
leave  in  direction  ci,  and 

P(j^|^F<ifJ>  =  Probability  that  a  target 
arriving  at  <i,J-^  from 
direction  ot'  will  leave 
at  direction  a. 

<U>  These  transitional  probabilities  can  be 
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derived  on  the  basis  of  deployment  doctrain 
and  terrain  constraints  as  a  function  of 
time  and  stored  in  a  data  base.  Figure  2- 
Alternati vely,  the  development  process  can 
be  mediated  in  real  time  through  use  of  such 
higher  level  reasoning  as  Expert  Systems 
analysis.  Such  an  approach  allows  reasoning 
to  be  broken  naturally  between  numeric  and 
symbolic  processors- 

<U>  Lacking  higher  reasoning  or  terrain 
constraints,  the  following  deployment 
defaults  are  meaningful: 

Pg|l^<i,j>  =  a  value  reflecting  the 
general  suitability  of 
the  terrain,  and 

P^|g(i,J>  =  indicating  independence 

of  the  new  and  old  track 
directions. 

(U)  If  the  target  is  transversing  open 
terrain,  then  its  direction  will  likely  be 
determined  by  the  local  terrain  suitability. 
Thus  its  future  directional  probabilities 
can  be  given  on  homogeneous  terrain  as: 

<U)  ModggQ.CAbsCot-ct'))]  <1) 

where, 

P^cpi  =  the  probability  of  a  delta 

angle  turn  of  p*. 

(U>  If  the  target  is  moving  on  a  road,  then 
it  will  most  likely  stay  on  the  road  and  not 
reverse  its  direction.  Alternately,  it 
could  depart  the  road  for  open  terrain  at 
angle  o6  with  lesser  probabilities.  One  can 
rather  straightforwardly  write  an  equation 
for  this  case  similiar  to  that  of  Eq.  1. 


<U)  Velocity 

CU>  Just  as  terrain  and  doctrine  can  exert 
influence  on  the  target's  directional 
tendencies,  they  also  exert  powerful 
influences  on  the  target's  velocity. 
Velocity  is  re-expressed  here  as  that  time 
required  to  leave  one  point  on  the  grid  and 
arrive  at  another: 

^  =  Time  units  required  for 

a  target  leaving  (i,J>  at 
direction  cL  to  reach  the 
next  grid  point. 

This  time  variable  should  include  a  V 
adjustment  for  distance  differences  between 
adjacent  points  on  the  square  grid. 

CU)  The  variable,  ^<i  »J5,  is  an  excellent 
point  where  higher  level  reasoning  processes 
can  interact  with  and  control  the 
algorithmic  time  projection  mechanisms.  It 
is  algorithmically  straightforward  to  allow 
TjCi  ,J)  to  be  a  real  time  function  dependent 
upon  deployment  considerations  and  doctrine- 

(U>  TARGET  STATE  PROJECTIONS 

<U>  Two  forms  of  target  state  projections 
are  required:  spatial  diffusion  over  terrain 
and  velocity  prediction  over  time. 

<U)  Terrain  Diffusion 

(U)  At  any  moment  the  geometric  state  of 
the  target  is  given  by  its  location  Ci,J>, 
its  orientation  ot,  and  whether  it  is  stopped 
or  moving.  This  leads  to  the  following 
definitions: 

Jf<^>  =  Probability  of  arriving  at 

<i,J)  from  direction  ot  at 
time  t. 
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=  Probability  of  leaving 
Ci,J>  at  direction  ct  at 
t i me  t , 

Pg<i,j)  =  Probability  of  being  stop¬ 

ped  at  (irJ^  time  t, 

-  Probability  of  being  stop¬ 
ped  at  Ci,J>  at  time  t+1- 

CU)  The  probability  sum  of  a  target  being 
located  or  passing  through  a  grid  point  is 
constant  at  each  time  instance.  This 
results  in  the  following  equations  for  the 
probability  of  the  target  being  stopped  at 
grid  point  Ci,J)  at  time  t+1: 

<U>  P^(i,J)  -  (1  -  ^ 

(2) 

The  probability  of  the  target  leaving  point 
<i,J)  at  direction  ot  at  time  t+1  is: 

<U)  PLCi,J,ot)  =  Pct|s<i,J>  Ps<i,j)  + 

(3> 

These  two  equations  must  be  executed  at 

every  grid  point  <if j>  at  each  time  instance 

t- 

(U)  Prediction  Over  Time 

<U)  Target  velocity  effects  have  not  been 
taken  into  account  in  the  above  terrain 
diffusion  process.  As  the  target  leaves  the 
grid  point  Cif  J)  at  direction  ct,  time 
ir)j^<i,j)  will  elapse  before  it  can  reach  one 
of  the  eight  neighboring  grid  points.  This 
implies  that  the  leaving  probability, 
P|_^if  time  t  will  contribute  to  the 

arrival  probability, 

P^Ci',  j'  »t+T)j^Ci,  J),360*-ot)  . 

(U)  Observe  that  in  accordance  with  laws  of 
conservation,  the  total  target  probability 
over  the  grid  must  sum  to  one  at  each  new 
time  instance.  Thus  at  the  end  of  each  time 
prediction  instance  and  before  the  next 
terrain  diffusion  projection,  the  total  grid 
probability  must  be  renormalized  to  one. 

DATA  FUSION 

<U)  As  the  target  state  projection 
continues,  the  probabilistic  target  location 
likelihoods  will  diffuse  throughout  the 
terrain  grid.  The  target  location  will 
become  totally  lost  unless  new  intelligence 
report  updates  are  infused. 

(U)  At  any  time  instance,  the  intelligence 
analyst  may  want  to  see  a  probabilistic  plot 
of  potential  target  locations.  This 
involves  a  form  of  data  fusion  whereby  the 
target  state  data  is  reduced  to  location 
probabilities. 

CU)  Finally,  there  will  be  recurring  need 
to  task  available  sensor  resources.  This 


requires  time  sensitive  fusion  of  sensor 
coverage  characteristics  with  the 
corresponding  target  state  data. 

(U)  Intelligence  Updates 

CU)  Ideally,  information  would  arrive 
continuously  allowing  the  target  location  to 
be  known  at  all  times.  Unfortunately, 
sensors  must  be  tasked  based  on  their  next 
avai labl i 1 i ty,  and  then  the  resulting  data 
collection  analyzed.  This  takes  time  and 
there  is  no  assurance  that  the  target 
intelligence  collected  will  be  of 
significant  value.  Meanwhile  the  target 
will  be  slipping  away.  As  a  result  there  is 
a  high  premimum  placed  on  the  use  of  any  and 
all  arriving  information,  even  if  its 
arrival  rate  is  irregular. 


CU)  Assume  that  at  various  times  T|^, 
probabilistic  information,  Pj^Ci,J),  bearing 
upon  the  location  of  the  target  will  be 
received-  This  new  data  will  then  need  to 
be  fused  with  the  time  prediction  at  Tj^- 
This  is  most  easily  done  by  likelihood 
readjustment  of  the  time  prediction: 


CU)  P^Ci,J,rt) 
CU)  P^Ci,J) 


PlM^i»J>  PA<i»JrCC) 

. . . L - 

PNCi,J)  P§Ci,J> 
- E - 


C4) 

C5> 


where 
CU)  L 


conditioning  is  on  L: 

Ifj  « 


CU)  Figure  2  shows  the  imposition  of  the 
data  fusion  update  function  into  the  target 
state  projection  cycle  as  an  asynchronous 
modi f ier . 


CU)  Projected  Target  Locations 

CU)  The  predicted  location  of  the  target  on 
the  grid  at  time  t  can  be  given  as; 

CU)  PpCi,J)  =  P^  Ci,J)  +  y  P^Ci,J,ot)  C7) 


CU)  Sensor  Tasking 

CU)  Since  the  purpose  of  sensing  is  to 
locate  the  target,  it  is  reasonable  to  drive 
the  sensor  tasking  directly  from  the  target 
location  prediction  of  Eq.  7.  Considering 
imagery  as  the  sensor  of  primary  choice,  at 
each  time,  t,  it  is  most  likely  that  the 
sensor  can  only  examine  a  ground  footprint, 
F: 

CU)  FCi,J)  =  1  for  all  Ci,J)  grid  points 

within  the  sensor's  field 
of  view  centered  on  grid 
poi nt  Cu, v) , 

=  O  elsewhere. 

CU)  The  size  of  F  will  usually  be  less  than 
the  non-zero  region  of  probability  of  target 
location  on  the  grid.  Obviously  it  is  then 
of  interest  to  place  this  footprint  at  that 
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grid  location,  Cu,v),  that  maximizes  the 
imagery  sensor's  probability  of  target 
detection,  Pj-  This  can  be  done  by  the 
following  action: 


(U)  <u,v> 
<u,v) 


ArgMax 

u,v 

ArgMax 

u,v 


[  Pi  3 


(9) 


[  I  F<i-u, j-v> 
if  J 


PpCif  J>1 

CIO) 


CU)  RESULTS 

CU)  The  above  state  projection  and  data 
fusion  framework  is  presented  in  a 
mathematical  and  probabilistic  framework 
intended  to  be  tailored  to  meet  the 
particular  broad  area  search  problem  at 
hand-  The  following  two  case  studies  should 
be  useful  in  understanding  the  general 
working  of  the  framework - 
CU)  A  Simplified  Example 

CU)  Use  of  the  algorithmic  processes 
presented  above  is  illustrated  in  graphical 
form  in  Figure  3- 

UNCLASSIFIED 


3  HRS  LATER  +  1  MINUTE  & 
1  MESSAGE  LATER 


4  HRS  LATER 


Figure  3-  CU)  A  Graphical  Example 

CU)  At  time  zero,  a  target  vehicle  is 
found  at  the  end  of  a  road;  therefore,  the 
location  probability  is  set  to  one  there  and 
zero  elsewhere-  The  vehicle  is  expected  to 
move,  so  the  location  probabilities  are 
projected  forward  one  hour  - 


CU)  The  expected  target  location  now 
lies  within  a  teardrop  shape  with  minor  axes 
along  open  terrain  and  major  axis  along  the 
road.  The  location  probability  projection 
is  next  moved  forward  two  hours  into  the 
future-  At  this  time  a  message  is  received 
that  the  target  was  not  found  in  a  sweep 
corridor-  Meanwhile,  the  location 
probability  contours  have  continued  to 
expand  in  all  directions  and  have  begun  to 
follow  the  branch  of  the  road-  Note  that 
the  location  probability  function  has; 
however,  been  set  to  zero  within  the  sweep 
corridor-  This  increases  the  probability  of 
seeing  the  target  elsewhere- 

CU)  At  three  hours,  it  is  seen  that  the 
outer  contour  of  the  location  probability 
function  has  expanded  further  and  that  the 
sweep  corridor  of  one  hour  ago  is  starting 
to  fill  in. 

CU)  A  second  message  is  now  received  that 
the  target  has  been  found  on  the  right 
branching  road-  Instantly,  the  locaticMi 
probability  function  collapses  to  a  point 
probability  of  one  with  zero  value 
elsewhere. 

CU)  The  target  location/data  fusion  process 
is  never  really  completed  since  the  target 
can  continue  to  move-  Thus  as  the  location 
probability  projection  is  continued  forward 
to  four  hours,  it  is  seen  that  the  target 
location  is  again  becoming  probabilistic 
over  a  widening  region. 

CU)  In  this  simple  example,  there  was  no 
attempt  to  task  the  sensor  in  an  optimal 
manner.  Rather,  the  data  fusion  process  was 
limited  to  opportunistic  collection-  This 
lies  within  the  true  purpose  of  the 
techniques  espoused  above  —  extensibility 
to  various  broad  area  search  data  fusion 
scenarios. 

CU)  An  Expanded  Simulation 

CU)  A  larger  simulation  of  the  process  of 
maintaining  contact  of  relocatable  targets 
over  broad  areas  has  also  been  developed. 
Terrain  traf ficabi 1 ity  factors  incorporated 
include  primary  and  secondary  road  networks, 
terrain  elevation,  railroads,  hygrographic 
and  forestation  features  of  Figure  4. 
Figure  4  also  shows  the  initial 
probability  function  as  a  small  dot  at  the 
target  garrison-  A  rectangular  footprint 
shows  the  latest  imaging  sensor  check  for 
acti vity- 

CU)  Results  indicate  target  mobilization 
activity  and  the  projection  process  is 
activated.  Figures  5  and  6  show  the 
broadening  of  location  probability  as  a 
function  of  time.  The  brightness  of  the 
probability  function  is  a  direct  indication 
of  the  strength  of  the  location  estimate. 
□bserve  that  the  resulting  location 
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Figure  4.  <U)  Initial  Garrisoned 

Probability 


probabilities  mainly  follow  the  road  while 
being  blocked  by  the  presence  of  the  river. 

(U)  As  real  time  catches  up  with  the 
projected  time  of  Figure  6,  opportunistic 
radio  directional  finding  reports,  ellipses 
in  Figure  7,  place  the  target  generally 
away  from  the  river.  Fusion  of  these 
reports  with  the  location  function  of  Figure 
5  results  in  the  alteration  of  the  location 
probabilities  away  from  the  river- 
Reprediction  forward  again  to  the  projected 
time  of  Figure  6  now  results  in  a  modified 
re-expansion  of  the  location  probability. 
Figure  8. 
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UNCLASSIFIED 


Figure  5.  CU)  First  Time  Projection  Forward 
UNCLASSIFIED 


Figure  6-  CU)  Second  Time  Projection  Forward 


Figure  7,  CU)  Fusion  of  Intelligence  Against 
First  Time  Projection 

UNCLASSIFIED 


Figure  8-  CU)  Reprojection  of  Second  Time, 
Sensor  Tasking  Recommendation 
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<U)  Further  time  expansion  will  require 
increasingly  large  areas  be  searched.  As  a 
result  it  is  decided  that  an  imagery  sensor 
pass  be  tasked  before  location  becomes  yet 
harder-  The  next  available  sensor  type  is 
identified  to  the  algorithmic  engine  and  it 
centers  the  footprint,  shown  in  Figure  8, 
to  the  region  of  highest  target  acquisition 
probability.  The  analyst  accepts  the 
recommendation  and  the  sensor  is  so  tasked- 
Upon  later  collection  analysis  no  target  is 
found,  the  collection  ground  footprint  is 
cleared  of  probability,  and  the  remaining 
probability  is  increased-  See  Figure  8 
where  it  is  shown  that  the  algorithmic 
engine  makes  use  even  of  the  negative 
information  that  the  target  is  not  present. 

UNCLASSIFIED 


Figure  10- 


<U)  Third  Time  Projection  Forward 
and  New  Sensor  Tasking 


the  target  is  found-  This  results  in 
collapse  of  the  location  function  to  the 
target's  location.  Figure  11, 

<U)  Again,  even  though  the  target  has  been 
found,  the  situation  can  not  remain  stable 
as  the  target  continues  to  move-  Indeed, 
the  later  timeproject ion  of  Figure  12  shows 
the  target  location  uncertainity  is  again 
expanding  in  area. 

UNCLASSIFIED 


Figure  9- 


(U>  Fusion  of  Collection  Analysis 
Against  Second  Time  Projection 


<U)  The  situation  is  now  becoming  more 
serious.  Further  time  projection  shows  by 
the  time  of  the  next  possible  sensor 
retasking,  the  location  probability  will 
have  significantly  grown.  Figure  10-  Also, 
the  prior  collection  footprint  is  rapidly 
filling  in.  Had  there  been  pockets  of 
uncertainity  in  the  prior  image  collection 
analysis,  then  this  filling  would  have  been 
even  more  complete  by  this  time.  For 
instance,  stands  of  trees  will  normally 
obscure  many  relatively  small  regions  from 
sensor  view-  The  footprint  of  Figure  10 
would  then  have  contained  small  pockets  of 
non— zero  probability  which  would  have  more 
rapidly  filled  in  than  before. 


CU)  Again  a  new  image  sensor  tasking  is 
requested-  The  algorithmic  engine  easily 
places  the  sensor  footprint  on  the  main  lobe 
of  the  probability  distribution.  Figure  10- 
Si  nee  the  current  sensor  collection 
footprint  is  well  orientated  against  the 
main  lobe  the  recommended  tasking  is 
accepted . 


<U)  Upon  analysis  of  the  collected  image. 


Figure  11.  fU)  The  Target  Is  Reacquired! 

CU)  CONCLUSION 

CU)  In  the  kind  of  broad  area  search 
situation  envisioned  here,  the  problem  is 
never  ending,  but  the  techniques  outlined 
here  can  insure  best  use  of  all  available 
resources- 

CU)  An  algorithmic  engine  combining  target 
projection  and  data  fusion  functions  can 
almost  certainly  keep  better  track  of  the 
intricacies  of  the  broad  area  search  problem 
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Jl 


Figure  12.  CU)  The  Process  Continues.** 

than  unaided  human  initiative.  The  overall 
process  should  be  used  synergetically; 
however,  with  the  analyst  retaining  full 
control.  Various  projections  can  be  made 
into  the  future;  relative  confidences  can  be 
assigned  to  projections  and  to  new  incoming 
intelligence;  the  results  can  be  refused; 
sensor  resources  selected  and  reselected; 
and  alternate  sensor  footprint  optimizations 
remade  as  desired.  The  framework  is 
available. 
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Mass  raid  attack  scenarios  are  considered  viable  in  some  regions  of  the  worid.  In  these  scenanos,  thousands  of  hostile  aircraft  transit 
nanow  attack  corridors  rapidly  and  are  accompanied  by  severe  radar  electronic  countermeasures  (ECM).  Maintaining  an  adequate  air  picture 
based  on  conventional  active  radar  tracking  of  individual  aircraft  or  small  foimations  may  not  be  possible  m  this  type  of  environment. 

An  automated  mass  raid  tracking  technique  has  been  developed  which  uses  data  from 
(ESM)  systems  to  restore  the  air  picture  which  might  be  lost  to  radars  in  a  severe  ECM  environment.  The  technique  exploits  the  kinematic  and 
electronic  signatures  of  mass  raids  to  detect  and  track  raid  leading  edges  and  centers  of  mass  and  to  locate  raid  corridors. 

In  this  technique,  emitters  of  hostile  aircraft  which  are  expected  to  be  well  represented  in  the  leading  edge  and  center  of  mass 
of  a  mass  raid  attack  are  targeted  with  ESM.  Detections  on  emitters  associated  with  the  leading  edge  components  of  *e  raid  are  bu^l^  to 
form  cluster  reports  which  represent  angular  regions  of  significant  emitter  activity.  The  legation  and  extent  of  the  ‘f  f joT 
computed  by  triangulation  with  similar  information  told  in  from  a  cooperating  remote  ESM  system  via  tactical  digital  data  li^.  The  l^a 
and  extent  of  the  centers  of  mass  are  determined  in  a  similar  fashion  and  an  air  picture  is  constracted  by  combining  the  results  of  the  two  pro¬ 
cesses.  Leading  edge  history  trails  define  the  raid  corridors  over  time.  Ghosting,  although  possible,  rs  significantly  reduced  and  responds  well 
to  logics  which  discriminate  on  kinematic  criteria  and  geographical  constraints. 

Algorithms  for  this  technique  have  been  developed  and  implemented  in  a  simulation  of  an  ESM  system  in  a  large-scale  mass  raid  scenario. 
The  paper  describes  the  algorithm  and  presents  results  of  the  simulation. 
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INTRODUCTION 

Mass  raid  attack  scenarios  which  are  characterized  by  high 
target  densities  and  severe  electronic  countermeasures  (ECM)  may 
constitute  an  environment  which  would  seriously  impair  surveil¬ 
lance  systems  designed  to  provide  active  radar  tracks  on  individual 
targets.  Although  major  radar  electronic  counter  countermeasures 
improvements  may  be  an  important  part  of  a  long  term  solution, 
there  are  opportunities  to  address  the  problem  sooner. 

Methods  for  building  an  air  picture  which  employ  information 
from  other  surveillance  sensors  is  one  alternative.  Any  such  meth¬ 
ods  should  consider  the  problems  introduced  by  the  large  number 
of  targets  in  the  environment  and  the  proximity  of  those  targets  to 
each  other.  Mass  raid  attacks  represent  extreme  examples  of  closely 
spaced  multiple  targets.  Such  an  environment  places  enormous 
demands  on  any  tracker. 

This  paper  outlines  a  method  for  building  an  air  picture  in  a 
mass  raid  environment.  This  method  is  called  Strobe  Bundling.  It 
uses  information  potentially  available  from  passive  sensors  such  as 
Electronic  Support  Measures  (ESM).  It  employs  clustering  and 
triangulation  techniques  to  locate  raid  leading  edges,  centers  of 
mass  and  raid  corridors.  It  is  intended  to  provide  display  informa¬ 
tion  useful  for  broadcast  air  control  of  tactical  resources.  It  does 
not  attempt  to  maintain  individual  tracks  on  targets. 

This  paper  motivates  the  Strobe  Bundling  method  by  dis¬ 
cussing  a  mass  raid  environment  and  the  sensor  information  which 
may  be  available  in  it.  A  description  of  the  method  is  given  as 
well  as  simulation  results.  The  report  concludes  with  a  discussion 
of  suggestions  for  further  development. 

BACKGROUND 

Mass  raid  scenarios  can  be  characterized  by  the  example 
shown  in  figure  1 .  In  such  scenarios,  many  hundreds  of  hostile 
aircraft  transit  narrow  attack  corridors.  In  addition,  a  large  amount 
of  radar  ECM  accompanies  the  attacking  aircraft.  This  ECM  may 
take  various  forms;  ground-based  and  stand-off  jamming,  escort 
and  self-screen  jamming. 

Mass  raids  are  composed  of  aircraft  with  different  specific 
missions.  Typically,  these  missions  include  bombing,  fighter  escort, 
jamming,  chaff  laying,  ground  attack,  and  reconnaissance.  The 
nature  of  these  missions  lends  a  certain  structure  to  the  raid. 


Reconnaissance  aircraft  fly  at  high  speeds  and  elevations,  detached 
from  the  main  bulk  of  the  raid.  Chaff  layers  precede  the  bombing 
and  attack  aircraft  and  define  a  leading  edge  to  the  raid  components 
in  each  attack  corridor.  Bombers  and  their  fighter  escorts  comprise 
the  bulk  of  the  raid  in  each  corridor  and  indicate  the  raid  center  of 
mass. 


ESU  • 
SENSOR 


Figure  1.  MASS  RAID  SCENARIO 

The  concentration  of  hostile  forces  into  narrow  transit  corri¬ 
dors  is  an  attempt  to  break  through  front-line  air  defenses  by  over¬ 
whelming  numerical  superiority.  The  extensive  jamming  employed 
reduces  radar  effectiveness  to  a  point  where  tracking  of  individual 
targets  cannot  be  adequately  performed  to  provide  the  surveillance 
information  necessary  to  weapons  control  and  other  C“  functions. 

The  loss  of  adequate  radar  data  does  not  preclude  the  exis¬ 
tence  of  other  usable  sensor  inputs.  In  particular,  electromagnetic 
emissions  from  on-board  avionics  equipment  can  be  targeted  by 
passive  sensors  such  as  ESM.  Although  ESM  detections  do  not 
provide  range  estimates,  they  can  be  used  to  estimate  the  direction 
to  an  emitter.  Further,  ESM  systems  have  the  capability  to  measure 
parameters  of  detected  signals.  These  parameters  may  include  fre¬ 
quency,  pulse  width,  pulse  repetition  frequency,  and  modulation 
characteristics.  Such  parametric  information  from  a  detected  emitter 
can  then  be  compared  to  emitter  library  entries,  and  an  assessment 
made  of  the  emitter  type  made.  Emitter  type  estimates  can  be  used 
to  make  an  estimate  of  platform  identification. 

Although  distributed  ESM  sensors  can  be  used  to  perform 
cooperative-passive  tracking,  this  surveillance  method  requires 
sufficient  system  sensitivity  to  assure  that  cooperating  sensors 
detect  the  same  emitters.  Moreover,  the  large  number  of  emitter 
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reports  potentially  available  in  a  mass  raid  environment  can  create 
a  large  number  of  false  intersections  or  ghosts.  Effective  deghost¬ 
ing  requires  that  all  the  reports  from  cooperating  sensors  be  avail¬ 
able  for  processing.  This  processing  can  be  done  at  one  of  the 
sensors  or  it  can  be  done  at  a  central  processing  location.  Regard¬ 
less  of  where  it  is  done,  a  large  data  transfer  capability  would  be 
needed  to  support  the  sharing  of  information  on  individual  reports. 

The  approach  taken  to  mitigate  these  requirements  is  to  pro¬ 
vide  raid  infonnation  rather  than  target  information.  Raid  informa¬ 
tion  would  include  such  characteristics  as  the  location  of  the 
leading  edge  and  center-of-mass  of  the  raids,  physical  extent  and 
target  count,  and  estimates  of  the  movement  of  the  raids  through 
the  corridors.  Processing  of  individual  ESM  reports  is  greatly 
reduced;  most  processing  occurs  on  combinations  of  data  from 
individual  reports. 

ESM  SENSOR  MODEL 

The  ESM  system  modeled  in  this  study  is  a  general  purpose, 
wideband  passive  surveillance  sensor  for  determining  direction-of- 
arrival  (DOA)  to  a  target,  as  well  as  emitter  and  platform  informa¬ 
tion  on  the  targeted  signal.  Parametric  measurements  of  a  signal 
can  be  used  to  distinguish  between  two  emitters  even  if  the  emitters 
are  not  distinguishable  in  DOA. 

The  ESM  model  used  for  this  study  assumes  that  emitter 
reports  along  the  same  DOA  azimuth  are  resolvable  (by  means  of 
parametric  data  comparison)  and  that  the  emitter  class  of  each 
report  can  be  determined.  An  emitter  class  is  composed  of  emitters 
associated  with  aircraft  performing  similar  missions. 

STROBE  BUNDLING  ALGORITHM 

The  Strobe  Bundling  algorithm  collects  ESM  reports  from 
individual  emitters  into  histograms  and  separates  these  reports  into 
clusters  of  significant  report  density.  Thus,  data  is  twice  fused  to 
reduce  subsequent  processing:  once  when  individual  reports  are 
placed  in  histogram  bins,  and  again  when  raid  clusters  are  formed 
from  the  histogram. 

Raid  clusters  from  cooperating  ESM  sensors  are  then  triangu¬ 
lated  in  a  manner  similar  to  conventional  triangulation  of  individual 
reports.  By  triangulating  clusters  associated  with  different  emitter 
classes,  leading  edge  and  center  of  mass  estimates  of  the  raid  are 
obtained.  Since  many  individual  reports  are  replaced  with  very 
few  cluster  reports,  the  ghosting  problem  is  significantly  less  than 
would  be  encountered  using  individual  reports. 


Figure  2.  STROBE  BUNDLING 


ESM  reports  from  individual  targets  close  in  azimuth  are 
collected  into  angular  bins.  Once  binned,  there  is  no  requirement 
to  keep  track  of  individual  reports  and  so  individual  report  DOA 


information  is  replaced  by  a  frequency  count  for  each  bin.  A  histo¬ 
gram  is  produced  of  DOA  versus  number  of  ESM  detections  per 
bin.  Figure  2  is  a  stylized  illustration  of  the  binning  process.  Each 
bin  in  the  histogram  has  angular  width  a.  The  height  of  each  bin 
corresponds  to  the  number  of  ESM  detections  whose  measured 
DOAs  are  within  the  limits  of  that  bin. 

An  example  histogram  is  shown  in  figure  3.  The  X-axis  is 
DOA  to  scenario  aircraft  from  the  ESM  sensor.  DOA  is  referenced 
to  True  North.  The  Y-axis  is  number  of  emitters  detected. 
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Figure  3.  EMITTER  COUNT  HISTOGRAM 

Clusters  of  significant  target  density  are  formed  from  the 
histogram.  A  cluster  report  consists  of  a  cluster  azimuth  and  cluster 
boundaries.  The  cluster  azimuth  is  a  single  numerical  value  which 
represents  an  average  DOA  for  the  cluster.  Cluster  boundaries  indi¬ 
cate  the  angular  extent  of  the  cluster.  A  cluster  report  can  be 
thought  of  as  a  “thickened”  strobe;  one  report  representing  a 
potentially  large  number  of  ESM  reports. 

Calculation  of  cluster  azimuths  consists  of  four  functions: 

•  “Noise”  filtering 

•  Cluster  separation 

•  Cluster  boundary  determination 

•  Cluster  azimuth  calculation 

Each  of  these  functions  will  be  discussed  below. 

“Noise”  Filtering 

The  emitter  count  for  each  bin  in  the  histogram  is  compared 
to  a  threshold.  This  thresholding  distinguishes  low  emitter  count, 
noisy  bins  from  bins  containing  the  significant  portions  of  the  raids. 
The  threshold  value  used  is  a  function  of  the  largest  emitter  count 
found  in  any  bin  across  the  angular  field  of  view  and  an  empiri¬ 
cally  determined  constant.  Thus 

THRESHl  =  NOISECON  •  max  {fj  (1) 

where 

THRESHl  =  emitter  count  threshold 
NOISECON  =  constant 

fj  =  emitter  count  for  bin  i. 

Bin  j  is  a  “zero”  bin  if  fj  <  THRESHl.  If  the  emitter  count 
for  a  bin  exceeds  THRESHl  it  is  called  a  “non-zero”  bin.  Zero 
and  non-zero  bin  designations  are  used  for  cluster  separation 
processing. 
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Cluster  Separation 

Cluster  separation  is  determined  by  comparing  the  number  of 
consecutive  zero  bins  separating  two  non-zero  bins  to  a  second 
threshold,  THRESH2.  Thus,  if  bins  j  and  j  +  k  +  1  are  non-zero 
bins,  and  bins  j  +  1,  j  +  2,  .  .  j  -b  k  are  all  zero  bins,  bins  j 
and  j  +  k  +  1  are  considered  in  separate  clusters  if 
k  >  THRESH2. 

This  second  threshold  is  determined  by: 

THRESH2  =  SEP/a  (2) 

a  =  bin  width  measured  in  radians 

SEP  =  minimum  separation  distance  between 
raid  clusters. 

The  value  of  SEP  reflects  a  judgment  as  to  what  configuration  of 
targets  in  a  given  scenario  represent  different  raid  components. 

Cluster  Boundary  Determination 

Cluster  boundaries  indicate  the  angular  extent  of  the  raid 
components. 

They  are  calculated  from  the  periphery  bins  by: 

CLUSTBNDRY  =  a  •  j  (3) 

where 

CLUSTBNDRY  =  cluster  boundary  value 

J  —  histogram  bin  number  of  a  periphery 
bin. 

Cluster  Azimuth  Calculation 

The  cluster  azimuth  is  calculated  as  the  average  angular  value 
of  the  bins  in  a  cluster  weighted  by  the  emitter  counts  of  the  bins 
in  that  cluster: 

CLUSTAZ  =  a  [nINT  [5^]  ■  Mod^  (4) 

where 

CLUSTAZ  -  cluster  azimuth  for  a  given  cluster 
fj  =  emitter  count  in  bin  j. 

NINT  is  the  “nearest  integer”  operator.  The  nearest  integer  opera¬ 
tion  locates  the  center  azimuth  bin.  The  sums  in  the  numerator  and 
denominator  of  the  operator  argument  range  over  the  histogram 
bin  numbers  associated  with  the  cluster.  Note  that  the  -1/2  term 
positions  the  cluster  azimuth  in  the  middle  of  the  appropriate  center 
azimuth  bin.  The  calculation  is  modulo  the  number  of  bins  to 
account  for  clusters  which  straddle  the  discontinuity  in  the  bin 
labelling  scheme  that  occurs  at  True  North. 

COOPERATIVE  PROCESSING 

Cluster  reports  from  a  single  ESM  sensor  provide  raid  detec¬ 
tion  information  and  an  estimate  of  direction  to  the  major  raid  com¬ 
ponents.  Azimuthal  motion  can  be  estimated  from  cluster  azimuths 
calculated  at  different  times. 

Single  sensor  cluster  reporcs  suffer  however  from  the  same 
restriction  as  individual  ESM  reports  in  that  they  do  not  provide 
range  information.  Range  estimates  can  be  obtained  using  triangu¬ 


lation  if  cluster  reports  are  available  from  a  cooperating  remote 
ESM  sensor  via  a  tactical  digital  communications  link.  Triangula¬ 
tion  of  cluster  azimuth  values  estimate  the  raid  centroids.  Triangu¬ 
lation  of  cluster  boundaries  estimate  spatial  extent  of  the  raid 
clusters. 

It  is  important  to  note  that  ^he  amount  of  ESM  data  required 
to  be  transmitted  over  the  communications  link  is  drastically 
reduced  when  cluster  reports  (or  at  most,  histogram  information) 
instead  of  individual  ESM  report  data  is  sent.  In  addition,  the  num¬ 
ber  of  intersections  created  by  triangulating  cluster  reports  is  gener¬ 
ally  a  small  fraction  of  those  that  would  be  created  while  doing 
conventional  cooperative  ranging  with  individual  reports.  The  num¬ 
ber  of  ghosts  drops  approximately  by  the  difference  of  the  squares 
of  the  number  of  individual  reports  and  the  number  of  cluster 
reports. 

Raid  stmeture  can  be  exploited  by  the  histogram/cluster  anal¬ 
ysis  algorithm.  Aircraft  with  different  missions  require  different 
classes  of  emitters.  Modem  ESM  sensors  can  be  expected  to  distin¬ 
guish  and  sort  signals  from  different  classes.  Histograms  can  be 
constructed  for  each  set  of  class-sorted  signals.  The  cluster  reports 
created  from  each  of  these  histograms  can  then  be  combined  to 
provide  more  information  on  raid  stmeture  than  if  emitter  signals 
are  binned  without  regard  to  class. 

SIMULATION  RESULTS 

The  algorithm  was  tested  by  simulation  in  the  scenarios 
described  earlier.  Figure  4  shows  combined  results  when  each  ESM 
sensor  processes  emitter  reports  from  raid  leading  edge  and  center 
of  mass  signature  aircraft  separately.  Cluster  boundaries  are  not 
shown.  Instead,  the  intersection  of  cluster  boundaries  from  cooper¬ 
ating  ESM  sensors  are  shown.  The  quadrangular  sets  are  called 
raid  extent  boxes.  Cluster  azimuth  strobes  from  both  ESM  sensors 
within  the  raid  extent  boxes  are  also  shown. 

Figure  5  illustrates  the  set  of  leading  edge  raid  extent  boxes 
over  several  processing  updates.  Note  that  the  raid  extent  boxes 
effectively  trace  out  the  raid  corridors. 
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Figure  4.  LEADING  EDGE  AND  CENTER  OF  MASS  CLUSTER  REPORTS 


Figure  5.  RAID  CORRIDOR  DETECTION 
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CONCLUSION 

The  current  implementation  of  the  Strobe  Bundling  algorithm 
represents  a  preliminary  effort  to  address  the  mass  raid  surveillance 
problem  using  distributed  passive  sensors.  Further  refinement  of 
the  design  would  concentrate  on  making  the  algorithm  more  robust. 
For  example,  the  logic  involved  in  cluster  separation  uses  constant 
value  parameters:  the  emitter  count  threshold  constant  NOISECON, 
the  separation  threshold  THRESH2,  etc.  These  parameters  could 
be  allowed  to  vary  to  reflect  changes  in  the  environment.  Similarly, 
the  present  algorithm  bins  the  data  using  an  empirically  determined 
constant  bin  width.  This  constant  value  is  the  best  value  on  the 
average,  but  certainly  is  more  useful  in  some  geometries  rather 
than  others.  A  more  robust  design  would  be  to  bin  the  data  simulta¬ 
neously  into  two  histograms,  one  with  a  relatively  wide  bin  width, 
and  one  with  a  relatively  narrow  bin  width.  The  wide  bins  should 
more  clearly  indicate  clusters:  emitter  count  distributions  will  be 
more  pronounced.  On  the  other  hand,  finer  grain  estimates  of  clus¬ 
ter  azimuth  values  are  possible  with  smaller  bin  widths.  Results 


from  both  histograms  can  then  be  combined  in  a  complementary 
way. 

At  present,  all  processing  is  done  with  just  one  cycle  time 
worth  of  data.  Thus,  prior  results  are  not  considered  when  doing 
current  processing.  This  memory-less  design  can  result  in  inconsis¬ 
tent  results  from  update  to  update.  Here,  “inconsistent”  refers  to 
the  number  of  clusters  determined,  the  sizes  of  the  raid  extent 
boxes,  etc.  A  feedback  mechanism  which  checks  for  consistency 
with  past  results  would  improve  this  situation. 

A  major  area  of  further  investigation  is  in  determining  a  mea¬ 
sure  of  the  algorithm’s  performance.  This  effort  is  complicated  by 
the  lack  of  how  to  identify  features  of  the  raid  which  the  algorithm 
measures,  and  then  to  translate  these  features  into  numerical  val¬ 
ues.  The  definition  of  measures  of  effectiveness  of  the  algorithm 
is  crucial,  so  that  results  can  be  compared,  and  improvements 
identified. 
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ABSTRACT 

Mass  raid  attack  scenarios  are  considered  viable  in  some  regions  of  the  world.  In  these  scenarios,  thousands  of  hostile  aircraft  transit 
naiTOw  attack  corridors  rapidly  and  are  accompanied  by  severe  ECM.  Maintaining  an  adequate  air  picture  based  on  conventional  active  radar 
tracking  of  individual  aircraft  or  small  formations  may  not  be  possible  in  this  type  of  environment. 

Density  reconstmction  is  a  technique  for  combining  electronic  support  measures  (ESM)  detection  data  on  a  mass  raid  which  has  been 
collected  at  three  or  more  cooperating  ESM  platfonns  to  reconstruct  the  size  and  shape  of  the  raid.  The  data  are  processed  to  construct  a 
digital  representation  of  the  mass  raid  called  a  target  density  map. 

Density  reconstruction  is  based  on  computerized  axial  tomography  (CAT)  technology.  In  medical  applications,  CAT  scanners  measure 
the  summed  physical  density  of  an  object,  such  as  a  human  chest  cavity,  from  several  viewing  angles  to  reconstruct  the  size  and  shape  of  the 
object.  In  density  reconstruction,  ESM  detection  data  collected  from  several  viewing  angles  are  processed  to  measure  the  summed  ermtter 
density  of  a  mass  raid  and  reconstmct  its  size  and  shape. 

Algorithms  for  this  technique  have  been  developed  and  implemented  in  a  simulation  of  an  ESM  system  in  a  large-scale  mass  raid  sce¬ 
nario.  The  paper  describes  the  density  reconstruction  algorithms  and  presents  the  results  of  the  simulation. 
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INTRODUCTION 

In  this  paper  we  discuss  a  new  technique,  target  density 
reconstruction,  for  using  passive  angle-only  data  from  multiple 
sensor  sites  to  derive  the  location  of  large  numbers  of  airborne 
targets.  In  a  high  target  density,  intense  jamming  environment, 
the  tracking  of  individual  targets  by  radar  may  be  severely 
degraded,  and  passive  sensors  such  as  electronic  support  measures 
(ESM)  may  provide  the  only  target  data.  Cooperative  passive  track¬ 
ing  using  ESM  data  may  itself  be  difficult  when  many  targets  are 
present.  The  target  density  reconstruction  method  described  here  is 
designed  to  gain  the  maximum  possible  raid  location  information 
from  passive  data.  Instead  of  individual  target  tracks,  it  generates 
a  target  density  map  (TDM)  with  estimates  of  the  number  of  tar¬ 
gets  in  each  of  many  small  grid  cells  covering  the  surveillance 
area.  Internally,  target  density  reconstmction  is  similar  to  a  comput¬ 
erized  axial  tomography  (CAT)  scanner. 


RATIONALE 

Target  density  reconstmction  is  a  means  of  deriving  the  max¬ 
imum  amount  of  summary  raid  location  information  when  standard 
tracking  from  radar  and  ESM  has  been  degraded  and  ESM  cooper¬ 
ative  tracking  is  not  possible. 

The  envirorunent  for  use  of  target  density  reconstmction  is  a 
massive  air  attack  with  hundreds  of  aircraft  in  multiple  waves 
crossing  a  border  in  one  or  more  corridors.  The  invading  aircraft 
are  protected  by  jamming  of  the  defending  radars.  Figure  1  shows 
the  locations  of  aircraft  several  minutes  into  a  generic  raid  of  this 
type.  The  dense  raid  environment  is  described  in  detail  in  Ref.  1. 
The  intense  jamming  during  the  raid  could  seriously  degrade  the 
ability  to  radar  track  individual  aircraft. 

We  assume  that  a  netted  system  of  ESM  sensors  is  in  place, 
such  that  any  potential  aircraft  location  is  in  line  of  sight  to  at  least 
three  ESM  sites.  Each  ESM  system  can  detect  and  measure  the 
radar  and  other  signals  being  emitted  by  the  aircraft  in  its  field  of 
view.  The  azimuth  angle  of  arrival  (AOA)  to  each  target  at  each 
ESM  site  is  determined,  along  with  signal  parameters  such  as  radio 
frequency  (RE)  and  pulse  repetition  interval  (PRI). 

With  a  small  number  of  targets  it  is  generally  possible  to 
combine  the  AOAs  at  two  ESM  sites  to  triangulate  a  target,  and  to 


Figure  1 .  MASS  RAID  TARGET  LOCATIONS 

perform  cooperative  passive  tracking.  However,  this  form  of  track¬ 
ing  requires  correct  association  of  the  angle  strobe  at  one  ESM  site 
with  the  strobe  generated  by  the  same  emitter  target  at  the  other 
ESM  site.  With  N  targets  to  be  tracked,  N  strobes  will  be  gener¬ 
ated  at  each  of  two  ESM  sites,  leading  to  N^  strobe  crossings. 

Each  crossing  is  a  possible  target  location,  but  only  N  of  them  are 
correct.  The  remaining  —  N  crossings  are  called  “ghosts.” 

With  data  fixim  three  sites,  the  ghosting  problem  can  be  even 
worse,  with  on  the  order  of  crossings  generated.  Figure  2  shows 
the  strobe  pattern  produced  at  three  passive  sensor  sites  by  the 
dense  raid  shown  in  figure  1 . 

In  less  dense  target  situations,  the  ESM  signal  measurements 
can  be  used  to  sort  out  the  valid  strobe  pairings.  But  with  the  large 
number  of  targets  being  postulated  here,  signals  from  different 
emitters  may  overlap  in  their  measured  parameters,  becoming  indis¬ 
tinguishable  to  the  ESM  system.  Emitters  may  also  change  their 
operating  characteristics  between  the  time  of  detection  at  one  ESM 
site  to  the  time  of  detection  at  another,  making  cross-correlation 
difficult  or  impossible.  For  example,  the  ESM  system  may  detect 
a  signal  once  each  10  or  more  seconds,  while  the  target  emitter  is 


changing  RF  and/or  PRI  many  times  a  second.  For  these  reasons, 
cooperative  tracking  by  ESM  sites  may  be  degraded  in  a  dense 
raid  environment - 


Figure  2.  STROBES  AT  THREE  ESM  SITES 


The  method  of  target  density  reconstruction  is  intended  to 
fill  the  gap  left  by  the  missing  radar  and  cooperative  passive  track¬ 
ers,  by  providing  an  estimate  of  the  general  raid  situation  in  as 
much  detail  as  possible.  Although  individual  tracks  are  not  derived, 
the  target  density  map  gives  an  estimate  of  the  number  of  targets 
in  each  10-km  by  10-km  grid  cell.  Figure  3  shows  a  TDM  gener¬ 
ated  directly  from  the  true  target  locations  given  in  figure  1.  The 
goal  of  an  estimated  TDM  using  ESM  measurement  data  would  be 
to  come  as  close  as  possible  to  figure  3 .  The  accuracy  of  the  esti¬ 
mated  TDM  depends  on  the  number  of  ESM  sites  measuring  the 
grid  cell  and  also  on  the  accuracy  of  the  ESM  system’s  measure¬ 
ments  of  AOA  and  signal  parameters.  The  method  makes  the  full¬ 
est  possible  use  of  the  available  input  data. 


Figure  3.  TDM  TRUE  DENSITIES 


TARGET  DENSITY  RECONSTRUCTION  PROCESSING 
Sorting  of  Signals  into  Subgroups 

The  use  of  ESM  data  for  target  location  depends  critically 
upon  the  ability  of  the  ESM  system  to  discriminate  between  emit¬ 
ters  based  on  signal  characteristics.  When  signal  features  are  mea¬ 
sured  and  data  from  different  ESM  sites  are  combined,  reports 
with  equal  signal  parameter  values  can  be  processed  as  a  separate 


group.  This  breaks  up  one  large  data  association  problem  into 
many  smaller  such  problems,  each  of  which  can  be  more  easily 
solved.  The  ideal,  of  course,  would  be  to  have  only  one  ESM 
report  per  site  in  a  group,  which  allows  for  cooperative  tracking. 
If  the  number  of  targets  in  the  group  is  small  enough,  relative  to 
the  angular  resolution  of  the  ESM,  discrete  tests  can  eliminate 
many  ghost  intersections.  The  principal  test  rejects  a  strobe  inter¬ 
section  from  two  ESM  sites  which  is  not  corroborated  by  a  strobe 
from  a  third  site.  Other  more  complicated  tests  can  be  employed, 
but  are  not  discussed  in  this  paper. 


Density  Reconstruction 

Target  density  reconstruction  is  an  application  to  ESM  data 
of  the  general  density  reconstruction  technique  which  has  been 
successfully  applied  in  such  fields  as  radio  astronomy  and  comput¬ 
erized  axial  tomography  (CAT)  scanning  (see  Refs.  2  and  3).  The 
technique  is  basically  a  way  of  recreating  the  distribution  of  the 
contents  of  a  region  when  all  that  is  known  about  the  region  are 
the  sums  of  that  content  along  lines  through  it.  There  are  two 
essential  input  requirements  which  must  be  satisfied  for  density 
reconstruction  to  work  in  the  target  problem.  The  first,  which  can 
be  satisfied  by  ESM  systems,  but  not  by  some  other  passive  sys¬ 
tems,  is  that  the  number  of  emitters  be  counted  along  each  angle 
of  arrival.  The  second  requirement  is  that  ESM  data  be  available 
from  at  least  three  sites.  The  count  data  from  only  one  site  are  not 
enough  to  perform  any  reconstruction,  and  two-site  reconstruction 
is  not  generally  useful.  Once  data  from  three  or  more  sites  are 
combined,  however,  density  reconstruction  can  give  a  good  picture 
of  emitter  location  densities. 


We  now  describe  the  computation  method  and  apply  it  to 
the  case  of  a  dense  raid  scenario  with  ESM  data.  We  also  point 
out  features  of  the  dense  raid/ESM  problem  which  make  ESM 
density  reconstruction  different  from  that  in  a  CAT  scanner  or  other 
typical  applications.  We  will  concentrate  on  the  case  of  three  sites. 
Processing  is  similar  when  more  sites  are  present. 


A  detects  A|  targets  In  sector  I 


Figure  4.  SECTOR  EQUATION 


In  the  ESM  density  reconstruction  problem  the  inputs  are 
the  number  of  emitters  detected  in  each  angular  sector,  for  all  the 
cooperating  sites.  Examine,  for  example,  the  ith  sector  with  respect 
to  site  A  (see  figure  4).  The  borders  of  sectors  centered  on  other 
sites  cut  across  sector  i  and  divide  it  into  many  small  odd-shaped 
pieces  Cjjk,  called  calculation  cells.  Here,  C^^  is  the  calculation 
cell  cut  out  by  sector  i  from  A,  sector  j  from  B,  and  sector  k  from 
C.  If  Xijk  is  the  number  of  targets  in  the  calculation  cell  and 
site  A  has  measured  A;  targets  in  sector  i  then 


A,  =  (1) 

(j  >k)eSjj^i 
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In  the  equation,  S^i,  is  the  set  of  pairs  (j,k)  such  that  is  a  cal¬ 
culation  cell  in  sector  i  of  site  A.  Each  sector  defines  one  such 
equation  relating  calculation  cell  target  counts  to  an  angle  sector 
count.  The  technique  of  density  reconstruction  is  to  solve  for  the 
Xjjfc  using  all  sector  equations  (Eq.  1)  for  all  sites.  If  there  are  three 
sites  and  n  angle  sectors  about  each  site,  this  means  that  3n  equa¬ 
tions  are  defined.  Only  3n-2  of  the  equations  are  independent, 
and  there  are  in  general  many  more  than  3n-2  calculation  cell 
unknowns  Xp,  so  the  system  of  equations  Eq.  1  is  badly 
underdetermined . 

Three  steps  are  used  in  ESM  density  reconstruction  to  reduce 
the  size  of  the  problem  and  find  a  reasonable  solution.  The  first  is 
to  notice  that,  being  a  count  of  targets,  no  X^^  could  be  negative. 
The  Xyk  must  then  not  only  satisfy  Eq.  1  but  also 


generates  a  particular  density  solution  {XsjJ,  where,  as  above,  Xp 
is  the  count  in  cell  Cij^.  The  number  of  configurations  having  the 
same  density  {Xj^^}  is  the  multiplicity 


W  =  -  (4) 

n  (Xp’) 

ijk 

Since  we  assume  that  each  configuration  is  equally  probable, 
the  likelihood  of  a  given  density  solution  {Xy^}  can  be  directly 
calculated,  since  it  is  just  the  multiplicity  W  of  configurations  hav¬ 
ing  that  density  divided  by  the  total  number  of  configurations, 
i.e.. 


Xijk>0  forallXijk  (2) 

This  step  reduces  the  number  of  possible  solutions,  but  does  not 
provide  a  unique  one. 

The  second  and  critical  step  uses  the  fact  that  many  angle 
sectors  contain  no  emitters  at  all,  even  in  a  dense  raid  scenario.  If 
a  platform  emitter  count  is  zero  in  a  sector,  then  all  the  computa¬ 
tion  cells  in  the  sector  must  be  empty  of  emitters,  and  their 
must  be  zero.  These  X^j,  and  the  sector  equation  can  now  be  elimi¬ 
nated  from  further  processing.  Since  many  X^ji^  are  eliminated  and 
only  one  equation  is  lost,  this  step  improves  the  balance  of  equa¬ 
tions  against  unknowns  as  well  as  reduces  the  computational  load. 
If  the  ESM  system  has  been  able  to  divide  the  strobe  data  into 
several  smaller  data  sets,  then  the  improvement  from  step  two  is 
even  sharper  than  when  all  strobes  are  processed  together.  This  is 
because  fewer  targets  and  AOA  reports  allow  for  additional  elimi¬ 
nation  of  computation  cells. 

Even  after  steps  one  and  two  the  system  defined  by  Eq.  1 
and  Eq.  2  is  still  badly  underdetermined,  with  no  unique  solution. 
The  problem  is  then  to  find  the  best  solution  from  among  many 
possible  ones,  on  the  basis  that  even  if  that  solution  does  not 
exactly  match  the  true  emitter  locations  it  will  be  close  enough  to 
be  useful.  Two  criteria  have  been  proposed  frequently  as  ways  to 
select  a  best  solution.  Each  algorithm  is  implemented  in  the  real 
number  system,  with  results  rounded  later  for  display.  The  firet, 
called  the  algebraic  reconstruction  technique  (ART),  is  an  iterative 
quadratic  optimization  which  converges  to  the  smoothest  solution, 
i.e.,  the  one  which  minimizes 


(3) 

ijk 

where  X  is  the  average  value  over  all  the  Xp.  See  Ref.  3  for 
details  on  ART.  An  update  generated  by  ART  does  not  automati¬ 
cally  satisfy  the  non-negativity  constraint  (Eq.  2)  on  the  solution. 
This  constraint  can  be  explicitly  coded  into  the  iteration  by  setting 
to  zero  those  Xp  which  would  otherwise  be  set  to  negative  values. 
ART  was  the  algorithm  used  in  the  first  commercial  CAT  scanner, 
but  has  given  consistently  poor  results  in  ESM  and  in  the  study 
(Ref.  4),  where  three-view  reconstruction  methods  were  compared. 

The  second  criterion  for  choosing  a  best  density  solution 
from  the  set  of  possible  solutions  used  a  probabilistic  approach  to 
choose  a  maximum  likelihood  solution.  The  reasoning  of  Jaynes  in 
Ref.  5  is  applied  to  the  target  problem  as  follows. 

Suppose  there  are  N  calculation  cells  and  a  total  of  R  air¬ 
craft.  We  make  the  simplifying  assumption  that  any  “configura¬ 
tion”  or  specific  placement  of  aircraft  into  the  cells  is  equally 
probable.  This  assumption  corresponds  to  not  knowing  the  prior 
probabilities.  There  are  configurations,  and  each  configuration 


Prob  ({Xp})  =  W/N«  (5) 

The  target  density  algorithm  then  seeks  the  maximum  likeli¬ 
hood  density  solution  over  all  densities  which  satisfy  the  ESM 
measurement  constraint  equations.  As  usual  in  maximum  likelihood 
estimation,  it  is  convenient  to  solve  for  the  maximum  logarithmic 
likelihood  solution.  Since  in  a  given  reconstruction  the  numbers  N 
and  R  are  fixed  and  known,  the  maximum  likelihood  solution 
occurs  at  the  maximum  of  ln(W).  In  reconstruction,  the  number  R 
of  aircraft  is  large,  allowing  use  of  Stirling’s  approximation  for  R!, 
to  give 


ln(W)/R  «  -  2^  (Xp/R)  In  (^R)  (6) 

ijk 

Since  the  sum  on  the  right  is  the  entropy  H({Xp})  of  the 
vector  of  numbers  {Xp} ,  maximizing  ln(W)  is  equivalent  to  the 
well-known  maximal  entropy  solution  to  the  ESM  angle  sector 
equations.  With  K  measurement  constraint  equations,  the  distribu¬ 
tion  of  configurations  near  the  maximal  entropy  solution  is  shown 
by  the  entropy  concentration  theorem  (Ref.  5)  to  be  chi-squared 
with  k  =  N  -  K  -  1  degrees  of  freedom.  Thus  for  large  R,  the 
fraction  F  of  configurations  having  entropy  within  AH  of  the  peak 
value  is  given  by 


2RAH  =  x'  (1  -  F)  (7) 

Since  we  assume  that  each  configuration  has  the  same  probability, 
this  expression  then  can  give  an  estimate  of  their  distribution  near 
the  maximal  entropy  peak. 

The  maximal  entropy  solution  is  easily  found  by  using  the 
MENT  iteration  algorithm  which  converges  to  the  unique  entropy 
peak.  It  can  be  shown  (Ref.  6)  that  the  Xp  in  a  maximal  entropy 
solution  have  the  form  of  a  product 


Xp  EjEjE)j  (8) 

Then  each  sector  equation  can  be  written 

X;EiEjE^  =  Ei  J]EjE,  (9) 

(j,k)6SAi  0,k)eSAi  (j,k)6SAi 

and  a  solution  must  satisfy 


Ei 


A: 


Eea 

O'>0eSAi 


(10) 
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In  the  MENT  iteration  algorithm,  each  equation  (Eq.  10)  is 
used  in  turn  to  update  the  value  of  a  single  Ej  from  the  current 
values  of  the  Ej  and  E^.  The  unknowns  Ej,  Ej,  and  E^  are  initialized 
to  1.  After  three  to  five  cycles  through  the  entire  set  of  sector  equa¬ 
tions,  the  E’s  are  changing  by  small  increments,  and  the  iterations 
are  ended.  The  output  X^k  are  then  generated  from  Eq.  8.  With 
185  targets,  the  MENT  algorithm  required  from  10  to  20  seconds 
to  reconstruct  target  TDMs  on  a  VAX  11/750  system. 

The  density  reconstmction  algorithm  can  be  summarized  as 
a  setup  stage,  in  which  a  minimal  number  of  unknowns  is  found, 
and  a  second  phase  in  which  the  linear  equations  relating  the 
unknowns  to  the  data  are  iteratively  solved.  Since  the  variables 
Xyk  refer  to  odd-shaped  calculation  cells  in  the  surveillance  region, 
some  of  them  quite  small,  the  resulting  solution  must  be  translated 
to  a  regular  grid  before  being  given  to  the  user.  This  is  done  by 
adding  together  the  densities  of  the  calculation  cells  contained  in  a 
grid  box  and  integerizing  the  result  for  output.  If  a  cell  has  parts 
in  different  grid  boxes,  its  calculated  density  is  prorated  among  the 
boxes  according  to  area. 

Implementation  of  Density  Reconstruction 

The  two  main  parts  of  target  density  reconstruction  are  the 
definition  of  the  unknown  densities  to  be  calculated  and  the  itera¬ 
tive  solution  for  them.  We  now  examine  the  different  approaches 
which  can  be  taken  to  implement  the  density  equations  and 
unknowns  and  the  reasoning  which  has  led  to  the  present 
algorithm. 

The  most  difficult  part  of  density  reconstmction  is  the  defini¬ 
tion  of  the  variables  Xjjk  and  their  equations.  This  step  is  quite 
simple  in  CAT  scanners  and  other  applications  of  the  technique, 
where  a  fixed  2-dimensional  grid  is  used  to  define  the  areas  whose 
densities  must  be  calculated.  In  ESM  density  reconstmction  such 
grids  have  given  poor  results,  leading  to  the  use  instead  of  calcula¬ 
tion  cells  described  above. 

The  problem  with  using  a  grid  to  define  the  unknowns  in 
ESM  density  reconstmction  is  that  it  cannot  accurately  represent 
the  reconstmction  geometry  near  the  ESM  sites.  For  example,  if  a 
sector  angle  size  of  1  degree  is  used,  then  a  1-km  grid  size  is  ade¬ 
quate  for  areas  fiirther  than  57  km  from  any  site.  A  grid  cell  closer 
than  this  will  not  be  contained  in  a  single  sector  centered  at  the 
site  so  its  variable  X  must  be  allocated  to  more  than  one  sector 
equation,  leading  to  inconsistent  equations.  To  overcome  this  sam¬ 
pling  problem,  variable  size  grids  were  tried,  but  these  were  found 
to  be  veiy  computationally  expensive,  as  well  as  producing  large 
numbers  of  variables. 


Figure  5.  SAMPLING  IN  DENSITY  RECONSTRUCTION 


The  method  which  we  used  replaces  square  grids  with  more 
general  angle-sector  sampling.  In  the  grid  definition  processing,  a 
coarse  grid  of  equilateral  triangles  50  km  on  a  side  covering  the 
surveillance  area  except  at  points  near  the  ESM  sites  is  scanned. 
For  each  coarse  grid  point  X,  sectors  from  all  platforms  intersect¬ 
ing  the  region  near  X  are  checked.  If  none  of  these  sectors  has  a 
positive  emitter  count,  then  the  region  is  empty  and  is  dropped 
from  further  processing.  Otherwise,  a  fine  grid  is  set  up  near  X  in 
the  following  way.  Referring  to  figure  5,  suppose  that  Site  B  is 
closest  to  X.  Then  the  midlines  of  angle  sectors  from  Site  B 
through  the  area  define  one  direction  of  the  fine  sampling.  Sample 
points  are  placed  on  the  midlines  so  as  to  give  the  coarsest  possible 
sample  spacing  which  is  still  adequate  to  represent  all  sector  equa¬ 
tions.  Each  point  of  the  finer  grid  is  checked  for  consistency 
between  the  three  ESM  sites:  if  any  platform  has  detected  no  emit¬ 
ters  in  the  sector  containing  the  point  then  the  point  is  rejected. 
Otherwise,  it  defines  a  new  variable  in  the  equations. 


TARGET  DENSITY  MAP  EXAMPLES 

In  this  section  we  look  at  two  TDMs  generated  by  the  target 
density  reconstruction  algorithm.  Each  TDM  is  an  estimate  using 
the  same  target  locations  as  figure  1,  so  the  ideal  result  would  be  a 
TDM  as  close  as  possible  to  figure  3.  Angle  measurement  error 
was  not  modeled  in  this  simulation,  and  an  angle  sector  size  of 
one  degree  was  used.  The  stars  in  the  figures  show  the  locations 
of  the  three  ESM  sites. 

Figure  6  shows  a  TDM  generated  on  the  assumption  that  the 
input  ESM  signal  parameter  data  did  not  allow  the  signals  to  be 
sorted  into  subgroups  for  separate  processing.  This  corresponds  to 
the  most  difficult  conditions  for  use  of  the  density  reconstmction 
algorithm.  The  figure  shows  that  principal  target  groupings  have 
been  successfully  reconstmcted,  although  finer  detail  is  missing, 
especially  at  points  far  from  the  ESM  sites. 
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Figure  6.  TDM:  RECONSTRUCTED  DENSITIES 

Figure  7  is  a  TDM  generated  on  the  more  favorable  assump¬ 
tion  that  the  ESM  systems  could  divide  the  signals  into  9  sub¬ 
groups  for  processing.  With  groups  of  up  to  20  emitters,  the 
density  reconstmction  algorithm  was  able  to  give  a  detailed  picture 
of  raid  groupings.  The  TDM  also  accurately  shows  isolated  small 
groupings  of  targets.  As  an  area  increases  in  size  (e.g.,  50  km  by 
50  km),  the  total  estimated  target  count  there  gets  closer  to  the 
tme  value.  However,  the  estimated  counts  in  an  individual  box 
can  differ  significantly  from  the  corresponding  tme  value  as  given 
in  figure  3. 
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and  the  MENT  algorithm  to  select  a  best  TDM  solution  from  the 
set  of  possible  solutions. 

Target  density  reconstruction  could  keep  the  user  informed 
of  major  concentrations  of  air  forces,  and  is  intended  as  a  backup 
in  case  primary  radar  is  degraded,  and  if  other  sources  such  as 
ESM  cooperative  passive  tracking  are  not  available. 
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Artificial  Neural  Network  (ANN)  technology,  ascribed  as  the  sixth  generation  computer, 
is  progressing  rapidly  in  both  devices  and  architecture  designs.  ANN  s  are  being  applied  to 
a  variety  of  problems  with  particular  success  in  solving  well-defined  adaptive  pattern 

•  4- Thp  results  are  achieved  faster  and  in  some  cases  more  accurately 
than^exrs?in™e"c  and  symbolil  ap^roLhes.  This  paper  describes  what  ANN-s  are  and  how 
Ley  are  trained.  A  taxonomy  is  given  along  with  MN  fquatio  . 

Current  ANN  workstation  capabilities  are  summarized.  The  application  of  ^ed 

multi-sensor  ID  is  addressed  including  a  stereo  image  matching  ANN  and  a  hybrid  distributed 
approach  to  automatic  target  recognition  (ATR) . 
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1 » 0  INTRODUCTION 

Artificial  Neural  Networks  (ANN*s)  are 
computers  that  learn  how  to  solve  problems 
based  upon  sample  data  and  built-in  learn¬ 
ing  mechanisms.  In  other  words,  ANN*s  are 
trained  to  identify  on  their  own  the  key 
f®3^tures  which  enable  them  to  distinguish 
different  patterns.  An  ANN  can  learn 
on-line  in  real-time,  or  can  be  trained 

a  user  with  a  sample  training 
set.  The  ANN's  do  not  require  expert 
knowledge  representation,  logical  inferenc- 
ing  schemes,  statistical  algorithms,  or  a 
programmer  to  develop  and  code  a  solution 
to  the  user's  problem.  Also,  ANN's  do  not 
provide  step-by-step  explanations  as  to  how 
answers  are  achieved.  ANN's  do  require  an 
architecture  with  sufficient  capacity  and  a 
training  scheme.  ANN's  provide  a 
complementary  addition  to  conventional  von 
Neumann  processing  for  problems  requiring 
pattern-recognition-type  tasks. 

This  paper  provides  an  introduction  to 
what  ANN's  are  and  how  they  are  trained  in 
Sections  2  and  3.  Section  4  summarizes 
their  current  capabilities.  Potential 
applications  to  multi-sensor  ID  are 
presented  in  Section  5. 


2.0  WHAT  ARE  ANN'S? 

The  human  brain  has  roughly  10^^  to 
10  neurons  with  10  to  10  connections 
each.  We  learn  by  modifying  these 
connections  most  of  which  are  local. 

Neurons  work  in  milliseconds  and  relax  to  a 
solution  in  less  than  a  hundred  steps 
(i.e.,  solutions  in  less  than  a  second). 

An  ANN  is  a  massively  parallel  inter¬ 
connected  network  of  simple  units  intended 
to  interact  similarly  to  biological  neural 
systems  (Ref  14).  As  such  ANN's  are 
characterized  as  feedback  dynamic  systems 
which  are  (Ref  4) : 


•  non-stationary :  time-independent 
node  responses 

•  non-convex:  numerous  local 
minima 

The  ANN  unit  activation  dynamics 
obey  an  autonomous  dissipative  system  of 
non-linear  differential  equations, 

_ i.  ”  ^i  ^^1 '  •  •  ‘  each  unit  i  activa- 

tion  level  X^.  (1) 

For  any  solution,  X(t),  X(t+c)  is  also  a 
solution  for  all  c.  These  paths  in  phase 
space  do  not  intersect  except  where  F.  (X)=0 
for  all  i.  Such  points,  X  ,  are  called 
critical  or  fixed  points,  8  critical  point 
is  asymptotically  stable  when  all  paths  in  a 
neighborhood  converge  to  it  over  time. 
Asymptotically  stable  critical  points  yield 
pattern  recognition  solutions  when  the 
appropriate  F.  have  been  learned  by  the  ANN. 
Three  techniques  that  have  been  entensively 
used  in  ANN  design  to  guarantee  convergence 
to  a  fixed  point  are  described  next.  First, 
Liapunov's  theorem  guarantees  that  a  criti¬ 
cal  point  is  asymptotically  stable  if  there 
exists  a  function  E  which  is  positive 
definite  and  whose  derivative  with  respect 
to  time  is  negative  definite  at  that  point. 
This  is  typically  achieved  via  a 
symmetrically  connected  ANN.  The  second 
approach  used  to  guarantee  convergence  is  to 
require  that  the  system  be  competitive 
(Ref  5)  , 

dF. 

_  ^  0  for  all  i^ij  (2) 

dx  . 

D 

(or  cooperative  by  reversing  the  sign  for 
all  Ref  12) ,  The  third  approach  is  to 

allow  only  feed-forward  connections  such  as 
illustrated  in  Figure  2-1.  The  trained  ANN 
classifies  an  external  input,  I,  by  settling 
initial  state,  X  ,  to  the  corresponding  ANN 
system  (i.e.  F(l,x))  fixed  point. 


non-linear;  node  response 
functions 

non-local:  nodes  take  outer  pro¬ 

ducts  of  their  inputs 


The  objective  in  training  the  ANN  is  to 
the  differential  equations,  specifi¬ 
cally  the  F^,  to  yield  the  correct  classifi¬ 
cation  (i.e.,  fixed  points)  for  any  allowed 
input.  In  first-order  ANN's,  the  F.  are 
parameterized  by  Long  Term  Memory  (ilM) 
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INPUTS  CAN  BE  INTENSITIES  PER  LOCATION 
PER  FREQUENCY  BAND 


HIDDEN  UNITS 

EACH  HIDDEN  NODE  APPLIES  SIGMOID  FUNCTION 
TO  WEIGHTED  SUM  OF  INPUTS  (ASYMMETRIC 
CONNECTIONS) 

FIRST  LAYER  PROVIDES  HYPERPLANE  DECISION 
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SECOND  LAYER  SEPARATES  CONVEX  SETS 

THIRD  LAYER  ARBITRARY  SETS 


INPUT  PATTERNS 
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OUTPUT  UNITS 

NETWORK  TRAINED  BY  DIFFERENCING  RESULTS 
FROM  TRUTH  CLASS 

Figure  2-1.  A  Feed-Forward  Neural  Network 
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TRAINING  FEEDBACK 


weights  (W. .)  associated  with  the  inputs  to 
node  i  froAi^ other  nodes  j  (including 
itself).  in  higher-order  ANN»s,  these  LTM 
weights  (W. -  depend  upon  combinations 

of  node  in^Stfe'/  There  are  two  approaches 
to  learning  these  LTM  weight  parameters  in 
ANN*s:  the  weights  maybe  trained  in 

real-time  (i.e.,  on-line),  or  they  may  be 
trained  may  be  trained  off-line.  Once  a 
fixed  set  of  LTM  weights  have  been 
determined,  the  ANN  can  be  used  as  a 
Content  Addressable  Memory  (CAM)  network. 

A  taxonomy  of  implemented  CAM's  (i.e., 

Short  Term  Memory  (STM)  dynamic  systems)  is 
shown  in  Table  2-1. 


Sufficient  fixed  points  need  to  be 
available  to  separate  and  store  all  the 
desired  memories.  Once  this  is 
accomplished,  partial  patterns 
incomplete  memories) 

the  initial  ANN  state,  X°.  The  ANN  then 
settles  to  the  associated  memory  fixed 
point. 

A  usefull  activation  function  is  the 
sigmoid  ^ 


Table  2-1.  Content  Addressable  Memory  (CAM)  ANN  Taxonomy  (Ref  9) 


SHORT  TERM  MEMORY  (STM)  DYNAMIC  SYSTEMS 

COHEN-GROSSBERG  (1983) 

_ I _ 


SHUNTING  FEEDBACK  NETWORK  (1973) 
(NON-LINEAR) 


ADDITIVE  (1967) 
(LINEAR) 


MASWNQ  FIELD 
(1978.  1986) 


I - ^ - - r  I 


The  general  STM  dynamics  model 
is  given  in  Figure  2-2.  Various  ANN  cm 
systems  may  be  defined  by  the  type  of 
units,  connectivity,  updating,  and  activa¬ 
tions  that  are  selected.  For 
Hopfield  associative  memory  additive  stm 
dynamics  equation  (Ref  13)  is 


dX, 


^i 


1 

dt 


where 


S  Wij  -  X^/R^  +  li  (3) 

is  the  weight  from  node  j  to 


node  i 

V.  is  the  G.(X.)  output  from 
^  node  j  ^ 

R.  is  the  parallel  sum  of  the 
^  resistances  1/W. • 

I.  are  the  external-^ inputs  to 
^  node  i 


where  the  .  and  6 .  effect  the  excitatory/ 
inhibitory  ind  threshold  firing  levels 
respectively.  The  t.  determines  the  gam 
(i.e.  slope)  of  the  Sigmoid  which  strongly 
effects  the  convergence  properties  of  the 
CAM.  Namely  for  large  t.  (i.e.  low  gain, 
,G’  small)  the  ANN  will  mote  likely  converge. 

In  fact,  convergence  is  guaranteed  (Rer 
■  whenever 


[I  ‘ 


(max 

i 


I)- 


1  (5) 


For  small  t,  (i.e.  higher  gain,  G  large) 
the  rate  of  ^convergence  is  faster.  However, 
there  may  not  be  a  fixed  point  and  the 
number  of  bifurcations  (i.e.  alternatir^ 
convergence  points)  tend  to 

decreasing  t.  causing  chaos  .  _ 

some  conclusfbns  may  be  drawn  from  the  time 
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INPUT  SIGNALS  FROM  OTHER  NODES 


SHORT  TERM  MEMORY  (STM)  DYNAMICS 

^  ■  A,(x,)  IB,(X,)  -  I;  W,j  Gj  (Xj))  i  F,(Xi) 
(GROSSBERG  CONTENT  ADDRESSABLE  MEMORY  MODEL) 
WHERE 

jm  TERM  MEMORY  (LTM)  DYNAMICS 

^  •  H.  <X,>  ID, A 


Vi  =  GjIXj)  OUTPUT  SIGNAL  TO  OTHER  NODES 

Figure  2”2.  Sample  First  Order  Neural  Network  Node 


convergence  points,  CAMs  with  unique  fixed 
points  have  been  preferred. 


3.0  HOW  ARE  ANN'S  TRAINED? 

A  taxonomy  of  learning  systems  is  given 
in  Table  3-1.  Two  useful  equations  for 
real-time  LTM  weight  dynamics  as  a  function 
G.  of  the  unit  inputs  from  the  jth  nodes 
arid  its  outputs  H^{X^)  are  as  follows  (Ref 


Hebbian  Learning  (With  Passive  Decay) 

^  =  -C.j  W,.  +  D..  G.(X.)  H.(X.)  (6) 

Competitive  Learning  (Gated  Decay  Hebbian) 
^  =  H.(X,)  I'-C,.  W,.  +  D,.  G.(X.)J  (7) 

Under  competitive  learning  the  weight 
vector  shifts  towards  the  stimulus  over 
time,  whereas  under  Hebbian  learning  the 
weights  increase  with  link  firing  caused  by 
the  stimulus.  Another  approach  is  dif¬ 
ferential  Hebbian  in  which  the  weights 
increase  with  the  rate  of  change  in  the 
link  firing. 


Off-line  learning  techniques  are 
divided  here  into  the  following  types: 


1.  Error  correction  (corrective 
feedback  per  output  node) 

2.  Reinforced  (overall  performance 

feedback) 

3.  Stochastic  (probabilistic  change 

using  energy  value) 

4.  Hardwired  (analytically  derived 

and  fixed) . 

A  popular  error-correction  technique  is 
back-propagation  (Ref  22,19,16)  which  is  a 
gradient-descent  technique  usually  applied 
to  first-order,  feed- forward  networks. 
Back-propagation  (i.e.,  delta-rule)  train¬ 
ing,  as  described  in  Figure  3-1,  uses  a 
steepest  descent  on  the  error-squared 

function  over  weight  space.  Namely, 


min  (e")  =  1/2 


_ 

dW^  . 
ID 


f  b  -  “(i 

pattern  P^  (8 


(error) ^  for  n^^  pattern  P^ 


gradient  change  for 


(9) 


Table  3-1.  Long  Term  Memory  (LTM)  Learning  System  Taxonomy 

REAL-TIME  (ON-LINE)  LEARNING 
(UNSUPERVISED) 

I  — - 1 

HEBBIAN  COMPETITIVE 

^  I  ^  (GATED  DECAY) 

DIRECT  SIGNAL  DIFFERENTIAL 


OFF-LINE  LEARNING 
(SUPERVISED) 


REINFORMED  HARDWIRED 

(OVERALL  PERFORMANCE  (ANALYTICALLY 
FEEDBACK)  DERIVED) 

I - * - 1 

SUCCESS/  SCALER 

FAILURE  PERFORMANCE 


STOCHASTIC  ERROR 

(PROBABILISTIC  CORRECTION 

CHANGE  USING  (OUTPUT  NODE 

ENERGY)  CORRECTIVE  FEEDBACK) 

I - - 1 

GRADIENT  QUADRATIC  DIRECT 

DESCENT  DESCENT  SEARCH 
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INTERCONNECT 
W||  WEIGHT  SPACE 


Figure  3-1.  Back-Propagation  (Delta-Rule)  Supervised  Learning) 


where 


Xi  =  X  W.jV. 


allows  feedback  within  each  layer,  and  a 
higher-order  ANN  is  of  the  form; 

(10) 

dX. 

1  _ 


This  has  been  shown  by  Lapedes  and 
Farber  (Ref  15)  to  be  mathematically 
equivalent  to  minimizing  a  Liapunov  energy 
function  for  a  master  N^-node  symmetric  ANN 
whose  weights  are  already  known  from  the 
original  ANN  being  trained  (i.e.,  the  slave 
ANN),  see  Figure  3-2.  Namely,  the  energy 
function  for  symmetric  master  ANN  is 


min 


N 

E 


i  #  j  /  ^  ^ 


^ijkl  ^iAl 


N 
-  Z 
if  j 


^ij^ij 


(11)' 


where  the  symmetric  weights  are; 


«ijki 


=  -  4 


(12) 


The  V.  .  fixed  points  yield  the  .  for 
slave  ^^network.  The  master  netwbrk 
convergence  rate  is  0 (N  )  which  is 
typically  slower  than  straight  back-pro¬ 
pagation.  However  with  analog  ANN's  it  may 
be  preferred  due  to  settling  time 
requirements  for  back-propagation. 


Pineda,  Ref  20,  has  extended 
back-propagation  ,  to  recurrent  and 
higher-orderTNlf' s .  A  recurrent  network 


dt 

Ai(X,)  k  (X.)-Z,.X  -k  Gj(Xj)-.Gk(Xk)] 

L  ^  (13) 


where  there  are  n  +  1  indices  and  the 

summations  are  over  all  the  indices  except 
i.  The  n  superscript  is  the  order  of  the 
correlation  of  the  network.  Pineda *s 
extension  applies  to  case  where  A. (X . )  -  -1 
and  B^(X^)  =  X^  +  and  basically  ^ets 

dw  .  .  n 

=  _c  d  (E  (Error) ^)  (14) 

dt  ^^^ij 

where  C  defines  a  time  scale  which  is 
suffici^tly  slow  that  X  is  always 
essentially  in  steady  state  (i.e.  weights 
change  adiabatically) .  The  solution  to  this 
linear  equation  for  the  weights  is  then 
recognized  to  be  the  fixed  point  of  another 
dynamical  system  of  transpose  size  which 
will  converge  if  the  original  system  con¬ 
verges  (Ref  1).  The  resultant  weight  update 
requires  O(N^)  operations  for  N  nodes. 

Second-order  schemes  (e.g.,  Ref  18)  try 
to  avoid  the  traditional  optimization 
problems  of  gradient  descent  by  using  the 


MASTER  COMPUTES 


WiJ  FOR  SLAVE 
FROM  VIJ  SOLUTION 


SI,AVE  ANM 


INTERCONNECT 
WEIGHT  W||  SPACE 


MASTER  ANN 


Figure  3-2.  Master  Network  Computes  Weights  for  Slave  Network* 
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N  N  N  M 

E  =  -1/2X;EWij  VjVj  -EIjVj  Gj^(V)dV  for  SYMMETRIC  W 

^  J  J  I  (HOPFIELD) 

•  SIMULATED  ANNEALING  IS  USED  TO  STIMULATE  THE  NETWORK  TO  ESCAPE  LOCAL 
MINIMA 

Figure  3-3.  Ann  with  Hardwired  Weights  Relaxes  to  Solution 


Hessian  of  the  error  function  with  respect 
to  weight  space.  Direct  search  techniques 
determine  a  descent  direction  based  upon 
numerous  performance  values  (e.g.,  typically 
N  +  1  points  in  N-space,  They  do  not  take 
analytical  derivatives  and  are  expected  to 
perform  better  for  very  rough  error  func¬ 
tions  by  passing  over  many  local  minima. 
This  approach  can  be  followed  by  a  gradient 
descent  approach  much  as  Hebbian  learning 
has  been  followed  by  gradient  descent  in 
some  applications. 

Hardwired  weights  have  been  used  when 
an  optimization  problem  has  been  formulated 
in  terms  of  an  unconstrained  minimization  of 
a  Liapunov  function.  The  Liapunov  function 
used  with  the  Hopfield  associative  memory  is 
shown  in  Figure  3-3  (Ref  13) .  This  approach 
has  been  used  in  the  Hopfield  solution  to 
the  traveling  salesman  problem  and  in  the  P. 
Castelaz  or  H.  Szu  (Ref  24)  solutions  to  the 
internetted  report  association  problem 
illustrated  in  Figure  3-4. 


Simulated  annealing  is  typically 
applied  to  keep  the  ANN  from  settling  into 
local  minima.  Such  an  approach  has  been 
proposed  in  preference  to  traditional 
approaches  for  very  large  problems  (e.g., 
N  >>  100) .  For  smaller  problems,  for 

example,  a  30-city  traveling  salesman 
problem  where  the  optimum  route  length  is 
4.3,  the  Hopfield  ANN  has  converged  to 
values  of  only  seven  or  less  whereas  a 
random  tour  typically  has  a  length  of 
twelve. 

Hardwired  weights  have  also  been  used 
for  non-symmetric  ANN's  to  solve  the  linear 
programming  problem.  The  problem  con¬ 
straint  matrix  becomes  the  weights  between 
a  column  of  the  primal  and  a  column  of  dual 
(i.e,  price)  variable  nodes  with  the  other 
variables  as  external  inputs  (Ref  6) . 

Reinforced  learning  is  applied  where 
error  correction  per  output  node  is  not 
available  and  only  a  success/failure 


O  -TRUE TARGETS 
0  -  GHOST  TRIPLETS 
X  -GHOST DOUBLETS 


Figure  3-4.  ANN  Hardwired  Weights  Have  Been  Applied  to 
Internetted  Association  Problem 
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response  or  scalar  performance  measure  is 
available.  A  general  equation  for 
reinforcement  learning  is 

(t+l)=W^j  (t)+n  (r(t)  -  e^)  e^j  (t)  ,  (15) 

where  n  is  a  constant  between  0  and  1  used 
to  regulate  the  rate  of  learning,  r  (t)  is 
the  success/failure  scalar  value  provided 
by  the  environment  at  time  t,  6.  is  the 
reinforcement  threshold  value  for  the 
output  unit,  and  e .  .  (t)  is  the  canonical 
eligibility  of  the  Weight  between  the 
and  the  jth  unit  at  time  based  upon  the 
probability  that  the  computed  output  value 
equals  the  desired  output  value  (Ref 
25,26) . 


4,0  WHAT  ARE  CURRENT  ANN  CAPABILITIES? 

A  description  of  the  top  commercial 
ANN  hardware  available  to  date  is  given  in 
Table  4--1.  Thus  current  ANN  hardware 
supports  thousands  of  nodes  which 
theoritically  can  yield  memory  capabilities 
also  in  the  thousands.  This  can  be  derived 
for  the  ANN  STM  given  by  Eq.  3  as  follows. 
(Ref  11)  .  Given  M  patterns  as  fix^ 
points  yields  NM  linear  equations  in  N  +  N 
unknowns  W.  .  and  R .  .  Requiring  asymptotic 
stability  ki  these ^  fixed  points  yields  an 
additional  2NM  piecewise  linear  inequali¬ 
ties  from  Gershgorin's  theoreom.  Thus 
independent  ANN  parameters  exist  yielding 
M  =  N+1  stable  equilibria  patterns  for  N 
nodes.  However,  the  observed  capacity  for 
many  ANN  systems  has  been  less  (e.g. 
approximately  .14N,  (Ref  13,15,22)). 

ANN  capacity  performance  is  con¬ 
tinually  improving.  In  fact,  ANN's  have 
demonstrated  pattern  recognition  perfor¬ 
mance  superior  to  existing  numeric  and 
symbolic  approaches  for  sonar  returns  (Ref 
8)  and  protein  strings  (Ref  21)  .  The 
payoff  for  ANN's  depends  upon  innovative 
ANN  STM  dynamics  and  LTM  training  designs 
implementable  in  cost  effective  "analog” 
hardware. 


5.0  PROMISING  ANN  MULTI-SENSOR  ID 
APPROACHES 

Multi-sensor  ID  is  the  key  missing 
technology  necessary  to  support  the  Beyond 
Visual  Range  (BVR)  attack  required  for  US 
fighters  to  win  against  a  superior  number  of 
threats.  The  multi-sensor  ID  function,  a 
sub function  of  the  multi— sensor  integration 
(MSI)  function,  uses  the  track  kinematics 
and  attribute  data  association  decisions  to 
identify  which  attributes  should  be  con¬ 
sidered  with  each  track  from  each  sensor,  as 
shown  in  Figure  5-1.  Multi-sensor  ID  uses 
these  inputs  plus  a  priori  data  to  generate 
the  MSI  target  ID  results  to  support  the 
pilot  interface  and  asset  management. 

A  key  problem  associated  with  multi¬ 
sensor  ID  has  been  that  the  individual 
sensors  have  difficulty  generating  their  ID 
declarations  with  accurate  error  descrip¬ 
tors/statistics  so  that  MSI  can  appro- ^ 
priately  combine  conflicting  and  uncertain 
ID  data.  The  multi-sensor  ID  problem 
requires  an  innovative  approach  to  overcome 
the  following  obstacles; 

•  long  solution  time  expected  to 
handle  sundry  multi-attribute  ID 
and  a  priori  data. 

•  inherent  instability  and  errors  in 
the  attribute  data  which  are  not 
well-characterized  (as  a  result, 
algorithms  for  clustering^  ID 
discriminants  are  ill-defined, 
necessarily  complex,  and  hard  to 
develop) ;  and 

•  requirement  for  fault  tolerance  in 
the  implemented  solution  technique 
and  hardware  architecture. 

ANN  approaches  appear  promising  due  to 
their  demonstrated  potential  for  the 
following: 

•  fast  settling  (milliseconds)  to 
"good"  solutions  since  the  long 
term  memory  is  imbedded  in  the 
unit  connection  weights  with  no 
von  Neumann  bottleneck  between 
memory  and  processor; 


Table  4-1.  Top  Commercial  Ann  Hardware  Available  (1988) 


- 

HAXIHUH  NUMBER 
OF  TOTAL 
INTERCONNECTS 
AND  NODES 
(MILLIONS) 

HOST 

INTER¬ 

CONNECTS 

PER  SEC 

H/WO  TRAINING 
(MILLIONS) 

WORKSTATION 

COST 

TRW  HARK  V 
(8  68020  BOARDS) 

5 

VAX 

5/10 

$70K 

SAIcS-1 

(DELTA  PROCESSOR) 

3.1 

PC/AT 

(80386/7) 

2.7/10 

$25K 

HNC  ANZA  PLUS 
(WEITEK  ACCEL) 

2.5 

PC/AT 

(ZENITH  386-80 

1.5/6 

$25K 

•AN  H-NODE  ANN  USES  UP  TO  N^  INTERCONNECTS 
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SENSORS 
TRACK  AND  ID 
PROCESSING 


9  self-organized  learning  which 

automatically  recognizes  ID  dis¬ 
criminant  patterns  based  upon 
training  data 

•  highly  fault-tolerant  since  it  is 

hardware  (not  software)  intensive 
and  redundant. 

These  inherent  ANN  properties  address  the 
technical  obstacles  to  successful 
development  of  a  multi-sensor  ID  avionics 
capability  identified  above.  The  ANN 
technology  is  ripe  now  for  multisensor  ID 
performance  demonstrations  using  one  of  the 
many  flexible  ANN  workstations  commercially 
available. 

Multi-sensor  ID  fusion  can  take  place 
at  many  levels  as  depicted  in  Figure  5-2. 
Higher  level  fusion  is  desired  to  minimize 
cost/complexity  by  allowing  individual 
sensor  processing.  Lower  level  fusion  can 
improve  performance  by  increasing  target 
detection  and  classification  confidence  and 
range.  As  always  the  level  of  fusion  design 
depends  upon  the  problem,  requirements,  and 
available  technology. 


One  problem  for  which  pixel  level 
fusion  has  been  suggested  (Ref  7)  is  the 
fusion  of  TV  and  FLIR  images.  The  TV  is 
strong  on  determining  the  boundaries  of 
objects  whereas  the  FLIR  photometric  data 
is  useful  for  extracting  features.  ANN*s 
are  potentially  useful  in  supporting  the 
corresponding  "boundary  contour  system" , 
"feature  contour  system",  and  the  "object 
recognition  system"  (Ref  10) .  Another  role 
for  ANN*s  in  pixel-level  fusion  is  in 
solving  the  image  co-registration  (or 
stereo  image  matching)  problem.  The  crux 
of  this  problem  is  to  determine  the  matching 
pixels  in  two  images  or  equivalently  to 
decide  what  the  disparity  is  between  one 
image  and  another  for  each  pixel.  A  poten¬ 
tial  ANN  to  solve  this  problem,  as  depicted 
in  Figure  5-3  (Ref  17)  ,  uses  many  layers 
each  representing  a  different  disparity 
level.  The  resulting  max  a  posteriori 
energy  function  for  this  recursive  ANN 
contains  image  compatibility,  smoothness  via 
Markov  Random  Fields  (MRF) ,  and  uniqueness 
contraints  on  the  solution.  Improvements  on 
this  basic  approach  are  given  in  Ref  3. 
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Figure  5-2.  Levels  of  Multi-Sensor  ID  Fusion 
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SENSOR  IMAGE 
INPUTS  V 


EXCITATORY 
CONNECTIONS 
(MRF  MODEL) 

INHIBITORY. 

CONNECTIONS 

(UNIQUENESS 

CONSTRAINT) 


ANN  LAYERED  INTO  DISPARITY 
LEVELS 

ANN  MAXIMIZES  PROBABILITY  OF 
DISPARITY  GIVEN  IMAGES 

A  MARKOV  RANDOM  FIELD  (MRF)  IS  USED  ON 
EACH  LAYER 


A  GIBB'S  DISTRIBUTION  USED  ON  LATTICE 
VIRT  THE  NEIGHBORHOOD  SYSTEM  (1,2, 3, 4) 
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Figure  5-3.  3-D  ANN  Image  Co-Registration 


ANN'S  can  raise  the  knee-of-the-curve 
design  level  towards  more  decentralization 
by  improving  the  target  classification 
speed  and  accuracy  of  individual  sensors. 
This  increased  parallelization  will  re(^c^ 
the  complexity  and  time  delays  associated 
with  more  centralized  approaches.  ANN 
technology  may  be  useful  as  a  preprocessor 
to  set-up  the  problem  parameters  for 
traditional  Automatic  Target  Recognition 
e.  Such  hybrid  systems  are 


Figure  5-4 .  An  example  of  such  a  decen¬ 
tralized  hybrid  application  of  ANN's  with 
numeric  and  symbolic  (e.g.  expert  system) 
processing  is  shown  in  Figure  5-5.  Here 
four  layer  feed- forward  ANN's  (with  recur¬ 
sion  within  layers)  are  applied  separately 
to  radar,  laser,  IR,  or  EO  image  or  spectra 
data  as  appropriate  and  trained  off-line  to 
support  target  ID.  Traditional  schemes 
(e.g.  log-polar  or  Fourier  transforms)  may 
be  used  to  accomplish  rotation,  scale,  or 


Figure  5-4. 


Complementarity  of  ANN  and  Conventional  Processing 
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Figure  5-5.  Distributed  Hybrid  Application  of  Artificial  Neural  Networks  with  Expert 

Systems  for  Cost  Effective  NCTR 
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ABSTRACT 

We  present  a  neural  network  model  for  sensory  fusion  based  on  the  design  of  the  visual/acoustic 
target  localization  system  of  the  barn  owl.  This  system  adaptively  fuses  its  separate  visual 
and  acoustic  representations  of  object  position  into  a  single  joint  representation  used  for  head 
orientation.  The  building  block  in  this  system,  as  in  much  of  the  brain,  is  the  neuronal  map. 
The  acoustic  processing  system  of  the  barn  owl  uses  them  to  form  a  map  of  space  on  the  structure 
known  as  the  external  nucleus  of  the  inferior  colliculus.  This  acoustic  map,  and  the  visual  map 
of  the  retina,  jointly  project  onto  the  optic  tectum,  creating  a  fused  visual/acoustic  represen¬ 
tation  of  position  in  space  that  is  used  for  object  localization.  In  this  paper  we  describe  our 
mathematical  model  of  the  process  which  maintains  the  registration  of  the  visual  and  acoustic 
inputs  to  the  optic  tectum.  The  model  assumes  that  the  acoustic  projection  from  the  external 
nucleus  onto  the  tectum  is  roughly  topographic  and  one— to-many,  whj le  the  visual  projection  from 
the  retina  onto  the  tectum  is  topographic  and  one-to-one.  A  simple  process  of  self— organization 
alters  the  strengths  of  the  acoustic  connections,  maintaining  a  small  region  of  strong  acoustic 
connections  whose  inputs  are  coincident  with  the  visual  inputs.  Computer  simulations  demon¬ 
strate  how  this  mechanism  can  account  for  the  existing  experimental  data  on  adaptive  fusion  and 
makes  sharp  predictions  for  experimental  test. 
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INTRODUCTION 

Neural  network  research  is  largely 
oriented  towards  solving  "high-level"  prob¬ 
lems  such  as  pattern  recognition,  catego¬ 
rization,  and  associative  memory.  Recent 
developments  in  this  field  have  produced 
powerful  new  architectures  and  algorithms 
for  solving  such  problems  (see  review  by 
Cowan  and  Sharp  (Ref.  1) ) .  Due  to  our  rel¬ 
ative  ignorance  of  how  such  problems  are 
solved  by  animals,  these  "high-level"  neu¬ 
ral  networks  are  only  loosely  based  upon 
known  principles  of  brain  organization  and 
function,  and  do  not  directly  correspond 
to  any  known  brain  structures.  However, 
there  are  other  equally  important  problems 
for  which  the  corresponding  brain  struc¬ 
tures  are  well  characterized.  Object 
localization  and  identification  is  one 
such  problem,  and  sensor  fusion  plays  an 
impotent  role  in  the  brain's  solution  to 
this  problem.  In  this  paper,  a  neural  net¬ 
work  approach  to  sensor  fusion  is  present¬ 
ed  that  is  based  upon  the  map-like  brain 
structures  that  solve  the  acoustic  object 
localization  problem  in  the  Barn  Owl.  Neu¬ 
ronal  maps  are  key  building  blocks  of  ner¬ 
vous  system  function,  ranging  from  percep¬ 
tual  classification  to  motor  control. 

These  structures  consist  of  locally  inter¬ 
connected  arrays  of  neurons  whose  response 
properties  vary  systematically  with  posi¬ 
tion  in  the  array,  thus  forming  a  map-like 
representation  of  information.  Computa¬ 
tion  is  achieved  through  transforming  the 
representation  from  one  map  to  the  next. 

The  fidelity  of  these  transformations  is 
maintained  through  dynamic  processes  of 
self-organization,  endowing  them  with  self- 
optimizing  and  fault  tolerant  properties. 
These  structures  are  linked  together  in 
modular,  hierarchical  processing  systems, 
that  employ  some  of  the  same  problem  solv¬ 


ing  approaches  used  in  technical  applica¬ 
tions,  such  as  sensor  fusion.  In  this 
paper  we  first  briefly  describe  the  stages 
in  the  chain  of  neuronal  processing  that 
generate  a  map  of  space  from  acoustic  tim¬ 
ing  cues,  and  adaptively  fuses  it  with  the 
map  of  space  derived  from  the  retina.  We 
then  describe  our  proposed  neuronal  mecha¬ 
nism  for  the  stage  of  visual/acoustic 
fusion  and  present  results  of  computer  sim¬ 
ulations.  This  mechanism  exploits  the 
coincident  signals  produced  by  an  object 
in  the  visual  and  acoustic  representa¬ 
tions,  using  adaptive,  non-linear,  neuron¬ 
like  processing  elements. 

RARN  nWL  VISUAL/ACOUSTIC  FUSION  SYSTEM 

Behavioral  and  physiological  studies 
have  revealed  that  owls  use  interaural 
intensity  cues  to  specify  the  elevation  of 
sounds,  and  interaural  timing  cues  to 
localize  the  azimuthal  direction 
(Refs.  2,3).  The  neuronal  processing  lead¬ 
ing  to  azimuthal  sound  localization  and 
visual  fusion  is  accomplished  by  a  series 
of  four  so  called  "computational  maps",  as 
shown  in  Fig.  1,  and  reviewed  by  Knudsen 
(Ref.  4) .  The  processing  for  elevation 
follows  a  similar  design  and  is  omitted 
from  the  present  discussion  for  clarity 
and  brevity. 

Nucleus  laminaris  (N.  lam.)  generates 
a  map  of  interaural  phase  delay  vs.  fre¬ 
quency  given  phase-locked  input  signals 
from  the  cochlear  nucleus.  The  central 
nucleus  of  the  inferior  colliculus  (ICc) 
transforms  the  N.  lam,  map  into  a  map  of 
frequency  vs  interaural  delay.  The  exter¬ 
nal  nucleus  of  the  inferior  colliculus 
(ICx)  transforms  the  ICc  map  into  a  map  of 
space,  forming  an  "acoustic  retina."  The 
acoustic  space  map  (ICx)  and  the  visual 
space  map  (retina)  are  fused  in  their 
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Figure  1.  Illustration  of  the  series  of  transformations  in  neuronal  representation  that  produce 
adaptive  visual/acoustic  fusion  in  the  object  localization  system  of  the  barn  owl.  Only  the 
azimuthal  acoustic  system  is  shown  for  clarity.  A  similar  parallel  series  of  transformations 
between  neuronal  arrays  computes  elevation. 


joint  projection  onto  the  optic  tectum. 

This  fused  map  of  object  location  is  then 
used  in  orienting  the  head  to  center  the 
object  in  the  visual  field  for  closer 
scrutiny. 

The  neuronal  processes  of  visu¬ 
al/acoustic  fusion  are  known  to  be  adap¬ 
tive  during  the  growth  period  of  the  owl 
(Ref,  5) .  This  is  essential  for  the  young 
owl,  for  during  this  time  the  distance 
between  the  ears  increases  severalfold, 
and  this  distance  is  a  critical  parameter 
in  computing  azimuthal  position.  Laborato¬ 
ry  experiments  have  shown  that  perturba¬ 
tions  to  either  the  visual  or  auditory 
transducers  (e.g..  goggles  or  ear  plugs) 
initially  cause  registration  errors 
between  the  auditory  and  visual  space 
maps.  As  a  consequence,  head  orientation 
driven  by  acoustic  cues  fails  to  center 
the  object  in  the  visual  center  of  view. 
However,  with  time,  fusion  is  reestab¬ 
lished,  and  proper  localization  behavior 
is  restored  in  a  continuous  manner. 

The  fused  sensory  map  of  space  in  the 
optic  tectum  is  also  a  motor  map  that  ori¬ 
ents  the  head  to  center  objects  in  the 
visual  field,  as  illustrated  in  Fig.  2. 


Displacement  vector 


Figure  2.  Representation  of  motor  command 
for  head  orientation  by  position  of  activa¬ 
tion  on  neuronal  map  in  head  centered 
retinotopic  coordinates. 


The  static  topographic  projection  of  the 
retina  onto  the  tectum,  and  the  fact  that 
the  eyes  do  not  move  relative  to  the  head, 
establishes  head  centered  retinotopic  coor¬ 
dinates  on  the  tectum.  Because  the  acous¬ 
tic  map  (ICx)  is  fused  with  the  visual 
map,  the  same  region  of  the  tectum  will  be 
activated  by  either  visual  or  acoustic  sig¬ 
nals  from  a  particular  location  in  space. 

A  vector  from  this  point  to  the  point  rep¬ 
resenting  the  center  of  the  retina  repre¬ 
sents  the  magnitude  and  direction  of  the 
head  movement  necessary  to  bring  this 
source  to  the  center  of  the  visual  field. 
Possible  neuronal-  mechanisms  for  this  have 
been  presented  by  Grossberg  and  Kuperstein 
(Ref.  6)  for  the  similar  case  of  saccadic 
eye  movements.  Multiple  objects  presum¬ 
ably  create  multiple  regions  of  activation 
on  the  tectum,  and  the  system  must  select 
a  single  target  for  the  head  orientation 
response.  Didday  (Ref.  7)  and  Arbib 
(Ref.  8)  have  presented  neuronal  models  of 
this  function  for  the  case  of  the  frog. 

NEURONAL  MECHANISM  FOR  SENSORY  FUSION 

Recent  work  by  Pearson,  Finkel  and 
Edelman  (Ref.  9)  demonstrated  a  solution 
to  a  problem  related  to  the  neuronal  map 
fusion  problem.  They  modeled  the  cortical 
map  of  touch  sensation  of  the  hand,  which 
contains  a  topographic  representation  of 
both  front  and  back  hand  surfaces.  The 
representation  of  these  two  surfaces  is 
not  fused.  Each  cell  in  the  map  responds 
to  stimulation  of  only  one  surface,  and 
cells  with  the  same  preference  are  clus¬ 
tered  into  regions  that  are  separated  from 
regions  with  the  opposite  preference  by 
sharp  borders .  Experiments  have  shown 
(see  references  within  Ref.  9)  that  the 
borders  between  these  regions  dynamically 
shift  so  that  more  highly  stimulated 
regions  of  the  hand  have  larger  regions  of 
representation  in  the  map  and  greater  reso¬ 
lution.  In  the  model,  each  cell  received, 
equal  numbers  of  connections  from  corre¬ 
sponding  regions  of  both  surfaces.  A  rule 
for  changing  the  strengths  of  the  connec¬ 
tions  strengthens  inputs  that  are  spatial¬ 
ly  and  temporally  correlated,  and  weakens 
those  that  aren't.  Since  the  two  surfaces 
are  rarely  stimulated  at  the  same  time, 
cells  weaken  their  connections  with  one 
surface  while  strengthening  their  connec¬ 
tions  to  a  small,  compact  region  of  the 
other . 

Our  model  for  fusion  in  the  tectum  is . 
a  simplified  version  of  this  model  of  the 
map  of  the  hand.  The  front  and  back  sur¬ 
faces  of  the  hand  correspond  to  the  visual 
and  acoustic  space  maps.  Fusion  is  pro¬ 
duced,  instead  of  segregation,  because 
stimulation  of  corresponding  regions  of 
the  two  input  maps  is  correlated  rather 
than  uncorrelated.  Fig.  3  is  a  pictorial 
representation  of  the  flow  of  signals  from 
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Figure  3.  Proposed  neuronal  architecture 
for  adaptive  registration  of  visual  (retina) 
and  acoustic  (ICx)  neuronal  maps  of  object 
position  in  the  tectum  of  the  Barn  Owl  fol¬ 
lowing  perturbation  to  the  visual  input 
(prism) . 

an  object  in  the  environment  through  the 
visual  and  acoustic  space  maps  to  a  cell 
(marked  with  a  filled  dot)  in  the  optic 
tectum.  The  light  lines  from  the  marked 
ICx  cell  to  the  tectum  delimit  its  diver¬ 
gent  region  of  projection,  while  the  heavy 
line  from  the  ICx  cell  to  the  tectum  repre¬ 
sents  the  functional  projection  created  by 
strengthening  that  subregion  of  the  total 
projection  and  weakening  all  others.  This 
divergent  projection  is  an  assumption  of 
the  model.  The  dashed  prism  indicates 
schematically  what  would  happen  if  there 
is  a  distortion  added  to  the  visual  field. 
Immediately  after  the  perturbation,  the 
visual  and  acoustic  maps  in  the  tectum  are 
out  of  register.  The  new  point  of  activa¬ 
tion  in  the  retina  (marked  with  an 
unfilled  dot)  immediately  leads  to  the 
activation  of  a  different  cell  in  the  tec¬ 
tum  (marked  with  an  unfilled  dot)  whereas 
the  acoustic  input  fires  the  same  cell  in 
the  tectum  as  before  (filled  dot) .  As  a 
result,  a  single  object  will  activate  two 
cells  in  the  tectum  instead  of  one,  and 
the  input  to  a  tectal  cell  will  be  half  of 
what  it  was  before  the  perturbation.  How¬ 
ever,  with  sufficient  correlated  stimula¬ 
tion  of  the  visual  and  acoustic  input 
maps,  the  connection  strengths  are  altered 
so  as  to  strengthen  those  acoustic  connec¬ 
tions  that  are  coincidently  activated  with 
the  visual  input,  and  to  weaken  the  origi¬ 
nal  acoustic  connections  that  are  no 
longer  activated  at  the  same  time  as  the 
visual  connections.  To  test  these  ideas  a 
simplified  computer  model  was  constructed. 

COMPUTER 

Fig.  4  shows  preliminary  results  of 
a  simulation  of  adaptive  fusion  following 
the  type  of  perturbation  to  the  visual 
input  described  in  Fig.  3.  These  drawings 
are  plots  of  the  input  connection 
strengths  from  the  acoustic  space  map 
(ICx)  onto  the  cell  at  the  center  of  the 
fused  map  (tectum) .  The  series  of  six 
drawings  shows  these  connections  as  ini- 
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Figure  4.  Simulation  of  reestablish¬ 
ment  of  visual/acoustic  fusion  follow¬ 
ing  visual  perturbation.  Arrow  indi¬ 
cates  site  of  acoustic  input  coinci¬ 
dent  with  visual  input  after  the 
perturbation. 


tially  assigned  (0),  after  1000  time  steps 
of  unperturbed  input,  (1000) ,  and  at  four 
successive  times  after  the  perturbation 
that  moved  the  center  of  coincident  input 
to  the  upper  right  as  indicated  by  the 
arrow. 

In  this  simulation  a  visual/acoustic 
stimulus  3x3  grid  points  in  size  was 
applied  in  a  random  sequence  over  the 
entire  input  grid,  coincident ly  activating 
topographically  corresponding  points  in 
the  space  maps  of  the  retina  (R)  and  ICx 
(X) .  The  tectal  cell  received  a  visual 
input  from  one  fixed  location  (RO)  that 
was  activated  when  the  stimulus  covered 
it.  The  stimulus  generated  input  was 
summed  to  yield  the  cell's  potential,  v, 
shown  in  Equ .  1 . 

V  =  cj  Xj  +  cO  RO  (1) 

where : 

cj  =  variable  acoustic  connection  strength, 
(0<c j<l) 

cO  =  fixed  visual  connection  strength, 
(c0=10) 

Xj  =  acoustic  input,  (Xj  {0,1},  l=on, 

0=off) 

RO  =  visual  input,  (RO  {0,1},  l=on,  0=off) 

The  acoustic  connection  strengths 
were  then  modified  according  to  a  simpli¬ 
fied  version  of  the  synaptic  rule  proposed 
in  Ref.  9,  shown  in  Equ.  2. 

cj(t  +  1)  -  cj(t)  =  k  s(v,ql,el)Xj 

(2) 

-  d  s  (V,  q2,  e2)  1 1  -  X j  1 

where : 

k  =  growth  constant,  (k=.05) 
d  =  decay  constant,  (d=.05) 
s  =  sigmoidal  function, 

s(v,q, e)  =  [exp ( (q-v) /e) +1]-1 
q  =  threshold  parameter,- (ql=9.5,  q2=16) 
e  =  sharpness  parameter,  (el=.01,  e2=.01) 


Parameters  ql  and  q2  set  the  thresh¬ 
olds  for  significant  strengthening  and 
weakening,  respectively.  Significant 
strengthening  of  an  acoustic  connection 
requires  that  its  input  be  active  (Xj=l) 
and  that  enough  other  strong  connections 
(whether  visual  or  acoustic)  be  active  so 
that  v>ql.  Weakening  an  acoustic  connec¬ 
tion  requires  that  its  input  be  inactive 
(Xj=0)  and  that  enough  other  strong  connec¬ 
tions  be  active  so  that  v>q2.  Given  the 
fixed  value  of  cO  (arbitrarily  chosen)  and 
the  stimulus  size  (chosen  based  on  simple 
considerations  of  scale) ,  ql  was  set  so 
that  weak  acoustic  connection  strengths 
would  grow  very  slowly  unless  they  were 
activated  coincidently  with  the  visual 
input,  and  q2  was  set  so  that  only  the 
coincident  activation  of  both  strong  visu¬ 
al  and  acoustic  inputs  would  produce  weak¬ 
ening  in  the  inactive  acoustic  inputs. 
Parameters  k  and  d  simply  determine  the 
time  scale  or  "smoothness”  of  the  simula¬ 
tion. 

The  reestablishment  of  fusion  is 
robust  to  changes  in  the  parameters,  as 
long  as  the  above  guidelines  are  met. 

Larger  stimuli  simply  enlarge  the  region 
of  the  strong  acoustic  connections.  Larg¬ 
er  values  of  e  soften  the  non-linearity  in 
s,  making  it  easier  to  strengthen  and  weak¬ 
en  connections,  but  do  not  significantly 
affect  the  results.  Regular  stimulation, 
in  which  the  stimulus  moves  over  the  input 
grid  one  point  at  a  time,  works  as  well  as 
random  stimulation.  Changes  in  the  thresh¬ 
old  parameters  affect  the  rate  at  which 
the  original  peak  decays  and  the  new  peak 
grows,  but  not  the  final  outcome,  because 
the  region  of  the  new  peak  has  the  advan¬ 
tage  of  a  fixed  visual  input.  Of  course, 
ql  must  be  less  than  the  voltage  due  to 
the  visual  input,  RO,  or  the  new  peak  can¬ 
not  grow,  and  q2  must  be  less  than  the  max¬ 
imal  potential. 

The  model  makes  several  biological 
predictions  that  could  be  tested.  The  mod¬ 
el  assumes  a  topographic,  divergent  projec¬ 
tion  from  the  ICx  to  the  tectum  (see 
Fig.  3) .  This  could  be  tested  with  vari¬ 
ous  anatomical  tracing  methods.  The  width 
of  the  divergence  sets  the  maximum  range 
over  which  registration  errors  can  be  cor¬ 
rected.  During  the  adaptation  to  the  per¬ 
turbation,  the  auditory  responsiveness  of 
cells  in  the  tectum  should  change  in  a 
characteristic  way.  A  new  region  of  audi¬ 
tory  responsiveness  should  appear  along 
with  the  original  region,  and  as  the  new 
region  gains  in  strength  the  original 
region  should  weaken,  eventually  vanish¬ 
ing.  At  first,  this  appears  to  be  in  con¬ 
trast  with  the  behavioral  result,  which  is 
that  the  localization  error  vector  slowly 
decreases  in  magnitude.  However,  it  is 
consistent  with  recent  findings  that  the 
motor  output  is  'determined  by  the  vector 
average  of  activity  on  the  tectum 
(Ref.  10) . 
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CONCLUSIONS 


Study  of  the  visual/acoustic  localiza¬ 
tion  system  in  the  Barn  Owl  has  disclosed 
a  potential  neuronal  mechanism  for  adap¬ 
tive  multi-sensor  registration.  Computer 
simulations  of  a  neural  network  model  of 
this  system  have  successfully  tested  the 
proposed  mechanism,  and  produced  predic¬ 
tions  for  experimental  testing.  Future 
work  must  determine  the  suitability  of 
this  method  for  technical  applications. 
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Abstract 


This  paper,  in  the  spirit  of  last  year's  artide, 
is  a  contribution  toward  a  unified  theory  of  data  fus¬ 
ion  as  an  integral  part  of  a  more  general  C^  theory.  A 
computationally  feasible  and  mathematically  sound  pro¬ 
cedure  is  proposed  here  for  the  combination  of  dispar¬ 
ate  information,  prior  to,  and  compatible  with,  ordin¬ 
ary  conditional  probability  evaluations.  This  is  based 
directly  upon  a  new  breakthrough  concerning  the  exten¬ 
sion  of  classical  probability  logic  to  a  full  condit¬ 
ional  logic. 

1 .  DATA  FUSION  AND  C^  PROCESSES 


1 .U  Qualitative  aspects. 

Previously,  in  [1]  a  general  approach  to  data 
fusion  was  outlined  within  the  context  of  C'^  processes. 

As  a  brief  review  (see  also  [2], [3])  and  a  modi¬ 
fication  of  past  efforts,  the  following  obtains: 

The  author  has  been  considering  C'^  processes 
from  the  generic  viewpoint  of  Interacting  nodes  of 
decision-makers, or  complexes  of  such.  These  interact¬ 
ions  or  "signals'*  may  be  actual  vectors  of  signals  con¬ 
taining  voluntary  or  leaked  Information  from  other 
nodes,  friendly  or  hostile,  or  they  may  represent 
fired  weapons,  for  example.  The  nodes  are  relative  in 
size,  but  whether  they  represent  one  or  a  group  of  in¬ 
dividuals,  they  possess  certain  common  characteristics. 
These  include  a  decision  structure  centering  around 
data  fusion  which  contains  detection,  hypotheses  form¬ 
ing  mechanisms,  algorithm  selections,  and  responses 
as  an  output.  The  nodes  also  are  represented  by  cor¬ 
responding  state  vectors  containing  all  pertinent  des¬ 
criptors,  such  as  equations  of  motion  and  location, 
number  of  individuals  or  supplies  and  their  relative 
importance,  damage  levels,  threat  levels,  and  esti¬ 
mates  and  other  knowledge  of  other  node  states,  friend¬ 
ly  or  adversary. 


Schematically,  the  following  simplified  situa¬ 
tions  hold  as  Figures  1  and  2  show: 


RESPONSE=WEAPON  FIRING 


RESPONSE' 
.SEND  SIG¬ 
NAL  FOR 


MEDIUM \  INFO, 
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Figure  1.  External  Dynamics  of  C 
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Figure  2.  Internal  Dynamics  of  C^  Processes :Sirnplifiecl 


The  basic  evolution  cycle  of  a  typical  node 
Is  described  in  the  following  figure: _ 
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RESPONSE 

|R’ 
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Figure  3.  Basic  Evolution  Cycle  of  a  Node  Due  to 
_ "Signal"  Processing  and  Response. 


The  components  of  a  typical  node  state  are  as 
in  Table  1: _ _ 


NODE  ^ 

STATE 

NODE  STATE 

PROPER 

THREAT  LEVEL 

#  OF  TROOPS 

#  OF  WP,I 

#  OF  WP,II 
IMPORTANCE 
SUPPLY  LEVEL 
EQ.  OF  MO. 
DAMAGE  LEVEL 

VECTOR i 

KNOWLEDGE 
,  PART 

ESTIMATES  OF 
OTHER  NODE 
STATES 

t  Table  1.  Components 

of  C^  Node  States.  1 

1.2.  Quantitative  aspects. 

The  next  step  following  the  qualitative  scoping- 
out  of  C^  processes,  including  data  fusion,  is  the  de¬ 
termination  of  the  corresponding  quantitative  descript¬ 
ion.  In  effect,  this  entails  choosing  both  an  appropri¬ 
ate  relational  syntax  and  a  numerical /semantic  evalua¬ 
tion  system.  Such  a  pair  is  called  an  algebraic  logic 
description -pair  (ALDP).  The  reader  is, most  likely, 
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familiar  with  the  most  common  ALDP:  probability  logic 
(PL)  where  the  appropriate  syntax  1s  the  structure  of 
a  boolean  (or  more  strongly,  sigma-)  algebra  R  of  events. 
Here,  the  usual  set  or  logical/propositional  operations 
hold:  Lnions(u)  or  dlsjunctions(v) ,  intersections(n)  or 
conjunctions (&) ,  complements  cr  negations  (O'),  meter- 
lal  implications  ((O'ul’O  (•)'v(--)»  commonly  de¬ 
noted  by*=>*»),  and  material  double  implication  or  mater¬ 
ial  or  logical  equivalence  ((•=>••}&(••=>•), commonly 
denoted  by««=>**  ).  For  purposes  of  simplicity  and  be¬ 
cause  the  author  has  propositional  logic  and  its  ex¬ 
tensions  in  mind,  the  common  notation  used  throughout 
this  paper  will  be  the  logical /propositional  inter¬ 
pretations-  but  these  can  all  be  immediately  convert¬ 
ed  to  the  set  notation  counterparts,  where  required. 

The  corresponding  semantic  or  numerical  evaluation  for 
PL  is  of  course  simply  the  choice  of  a  particular 
(joint)  probability  measure  (either  finitely  additive 
or  countably  additive,  if  need  be)  p:R  [0,1],  the 
last  symbol  denoting  the  unit  interval.  (See  [1]  and 
[4]  for  background.) 

As  pointed  out  in  [1]  and  elsewhere  ([4],Chpts 
1,  2.2.1),  other  processes  are  also  involved  in  estab¬ 
lishing  an  evaluation  of  a  situation  such  as  that  en¬ 
tailing  and  data  fusion  processes,  besides  the 
choice  of  ALDP.  For  example,  mentnl  imaging  and  cogni¬ 
tive  processes  play  an  important  role, as  do  natural 
language  formulations,  semiotics,  and  full  formal  lan¬ 
guage  /  formal  theory.  For  simplicity  only  the  latter 
will  be  considered  in  any  detail  prior  to  the  choice 
of  ALDP.  This  sequence  of  knowledge  flow  in  converting 
any  qualitative  description  into  a  quantitative  one 
can  be  summarized  in  the  diagram  in  Figure  4:  _ 

QUALITATIVE  PHASE 

PROBLEM/SITUATION  COGNITION —^NATURAL  LANGUAGE 
(C3  PROCESSES, E.G.)  FORMULATION 

QUANTITATIVE  PHASE  i* 

ALGEBRAIC  LOGIC  FULL  SEMIOTICS/ 

DESCRIPTION  PAIR:  FORMAL  ^ - STRINGS  OF 

(SYNTAX, SEMANTICS)  LANGUAGE/  SYMBOLS, 

FORMAL  DICTIONARIES 

THEORY 

Figure  4.  Knowledge  Flow  in  Describing  Situations. 


Examples  of  ALDP's  include: 

probability  loqic(PL)  :(boolean  algebra ,prob.  measure) 
fuzzy  logic(FL) : (browerian  lattice,  possibility  meas.) 
Dempster-Shafer  loglc(DSL) : (boolean  al gebra ,bel ief  ms.) 
classical  logic  (CL):  (boolean  algebra, 0-1  valued  ms.) 
(See  [1]  or  [4]  again  for  further  details.) 

In  the  case  of  C  processes,  an  Improved  full 
formal  language  description  has  been  developed  for  the 
dynamic  evolution  of  a  typical  node  state  vector 
[5],  replacing  previous  efforts  in  [l]-[3].  The  next 
section  describes  this. 

1 . 3  F^ull  formal  language  description/  theory  for 
node  evol utions. 

In  brief,  the  full  formal  language  description 
is  summarized  in  the  following  tables: _ _ 


EQUALITY  SYMBOL:  = 

CONSTANTS:  0 

DUMMY  VARIABLES:  a,B,Y 

SPECIFIC  VARIABLES:  N,R,S;  T  IMPLICIT  IN  N 

OPERATORS:  {)*A)'A)g’  (  I  )■  v.  DOM.  e 

GENERAL  AXIOMS:  FOR  ALL  a.B.Y.  AND  FOR  *  =  &.v: 
RING  STRUCTURE  FOR  &,v: 
a*B  =  6*a  ,  o*(6*y)  '  (o*b)*y  > 

a&0  =  0,  a  *  aV0,  aVfi  - 

a&(BVY)  -  {a&3)v(a&Y). 

Table  2a.  Formal  Language  Description  of  a 
_ C^  Node  Evolution:  Part  1. _ 


GENERAL  AXIOMS:  FOR  ALL  a,3,Y,  AND  FOR  *  =  &,v  : 
IMPLICATIVE/CONDITIONAL  STRUCTURE  FOR  &,v: 

(alfi)  =  a  ,  (a|B)  =  (a&B|3)  > 

(a&3lY)  =  (a[ 3&y)&(3 |y)^ 

(a*BiY)  =  {a|Y)*{P|Y)  , 

VO  =  n  . 
aED0M{rt) 

SUFFICIENCY  AXIOMS:  FOR  R  -  (N&N‘ 

(r'*"^|n'^&n)  =  (r'^'^In'^) 

N)  =  (N^^lR^S  N+) 

(N'^|S  &  R"&  R)  =  (n'^IS  &  N) 

(SIR'S  N)  =  (S|R') 

Table  2b.  Formal  Language  Description  of  a 
Node  Evolution:  Part  2. 


■&N"i--&Np): 


The  Sobols  in  Table  2  can  be  interpreted  as 
fo11ows,as  given  in  Table  3: 

■'  N  =  NODE  STATE  VECTOR  ,  T  =  NODE  STATE  STRUCTURE 
R  =  RESPONSE  VECTOR  ,  S'«  "SIGNAL"  VECTOR 

{  )*  =  POSITIVE  TIME  SHIFT  TO  NEW  PHASE 

(  )■  =  NEGATIVE  TIME  SHIFT  TO  OLD  PHASE 
{  )q  »  INITIALIZATION  OF  STATE  (TIME-WISE) 

(  1  )  =  IMPLICATION  OR  CONDITIONING 
&  =  AND  ,  V  =  OR  .  (  )'  =  NOT  (EXPLAINED  EARLIER) 
DOM  =  DOMAIN  OF  POSSIBLE  VALUES 
c  =  SET  MEMBERSHIP  RELATION  AS  USED  BEFORE 

Table  3.  Interoretations  of  the  Formal  Language 
for  C^  Node  Evolution. 


Note  that  N  and  T  above  can  be  partitioned  into 
subvectors  as  e.g.: 

N=  (#WP^  JU'P2.#W'’3>™0P,EQM0,lfrF0)  (1.1) 

T=  (DET.ALG.HYP.FUS, CONS, DEC)  (1.2) 

where  DET=detection  (or  not),  ALG=algorithiti  selection, 
HYP=hypotheses  formulation,  FUS=data  fusion, DEC=deci^- 
ion,  etc.  One  can  add  the  constraint  (1.3)  to  the  ax“ 
ioms  in  Table  2b: 

(R'^'^IT'^S  N'^)  =  (R'^'^lDEC'^S.  n'*').  (1.3) 

Using  similar  sufficiency  assumptions,  implicative 
chaining  in  Table  2b  shows  that 

(t'^|n‘^)  =  (dec'^|cons‘*'&fus'^&hyp'^&alg'^«iDet'^&n'^) 

&(  cons'*"  I  fus'''&hyp'*’&alg'‘'&det'*'&n''’) 

&  ( FUS'*'  I  HYP'''&ALG'''8,DET'''&N''') 
&(HYP'*'iALG'''&DET'''&N'*) 

C(ALG‘''|DET''’&m'^)4(DET'''1n‘^)  .  (1.4) 

Finally,  applying  the  usual  deduction  procedure 
to  the  axioms  given  in  Table  2,  yields  the  following 
theorem  (1 ,1 )describing  the  dynamic  evolution  of  a 
node  state  in  formal  language  terms: 


Theorem  1.1  (See  [5].) 


(N'*"''|N) 


\'here 


/R  e  D0M(R 

In'*'  c  dom(n''' 


Under  the  assumptions  in  Table  2: 

(  N'*'*' I  R''"*&N'^  )  S(  1 N"^ )  &  ( N"*  1 N ) ) 

(1.5) 

(1.6) 

(1.7) 

(1.8) 

(1,9) 

■ 


D0M(N  ) 

(n'^In)  =  V  ((n^|r‘&n)mr'|n))» 

R'eDOM(R') 


(r'^'^|n'‘')=  +v  ((r'^'''|t'*&n'^)«.(t*|n'^)), 

T  eD0M(T  ) 

(N'''|r'&N)  -  V  ((N'*|S&N)&(S|R'))  , 
SeDOM(S) 

ind  1  .  J.J. 

N  =  V  ((N  |N)&  N)  . 

NcDOM(N) 
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Q  =  ((alb)  &  (c|d))  V  (e|f), 


(2.3) 


Thus,  using  the  interpretation  In  Table  3,  com¬ 
patible  with  Figure  3, 

{r'*^'^|T^&N^)  =  response  following  processing,  (1.10) 

=  new  node  state  due  to  its  sending 
out  response  ,  (l  .11 ) 

(T^|N^)  *  processing  data,  (1.12) 

(N*'’^lN)  =  full  cycle  of  node  change  due  to 

"signals"  received,  over  all  poss- 
ible  processing,  and  responses,  (1.13) 

Thus,  If  PL  were  chosen  as  the  ALDP,  assuming 
only  stochastic  relations  are  involved  in  C'^  variables, 
then  Theoreml.1  reduces  to  the  more  familiar  form 


p{N 


"|N)=  /  p(N''‘'1r'  .. 

\H+eD0M(N+)  / 


(1.14) 


Or,  if  FL  were  chosen  as  the  ALDP,  assuming  only  fuzzy 
relations  are  involved  in  variables,  then  Theoreml.1 
becomes  under  semantic  evaluation 


poss(N 


max{  min(poss(N^^lR^^.N^ 
over  all  \  poss(N‘‘^lN 
R++eD0M(R++) , 

N'^eD0M(N+)  J 


ss(R^^In^) 

(1.15) 


One  could  also  choose  combinations  of  PL  and  FL  or 
other  ALDP's  in  the  evaluation  aspect.  (Again,  see  [4D 


In  turn,  utilizing  the  outputs  in  Theorem  1.1, 
together  with  the  choice  of  ALDP,  it  is  clear  that  the 
evolution  of  node  states  depend  on  the  determination 
of  the  relative  primitive  relations  given  as  suffici¬ 
ency  axioms  in  Table  2b.  Calling  each  possible  combin¬ 
ation  of  such  relations  for  each  side  j, friendly 
(j«l)  or  adversary  (j=2),  PRIM^  ,  one  can  establish 

a  loss  L(PRIM^ .PRIMg)  based  directly  on  Theorem  1.1, 

thereby  establishing  a  zero  sum  two"person"C  decision 
game  and  then  precede  to  analyze  the  game  for  values, 
least  favorable  strategies,  bayes  decisions,  minimax 
strategies,  etc.  (See  [5]  for  further  details,  where 
a  multidimensional  gaussian  linear  conditional  struc¬ 
ture  is  Imposed  upon  the  relations  yielding  feasible 
computational  forms.) 

Throughout  all  of  the  above  calculations,  data 
fusion  plays  a  central  role  (see  again  (1.4)  anc  (1.7; 
where  the  specific  quantitative  relations  depending 
upon  data  fusion  are  shown).  In  the  next  section,  mo¬ 
tivation  is  developed  for  a  systematic  approach  to 
the  fusion  proper  aspect  for  disparate  information  ar¬ 
riving  in  conditional  form. 


2.  NEED  FOR  A  CONDITIONAL  EVENT  ALGEBRA  IN 
EVALUATING  DATA  FUSION 


2.1  Introduction. 

Noting  that  many  of  the  relations  in  Table  2b 
and  subsequent  equations  are  in  conditioned  form,  con¬ 
sider  in  particular  the  basic  data  fusion  factor  in 

Q  =  (FUS“^|HYP'^&  ALG"^  &  DET'*’  &  n'*‘),  (2.1) 


noting  that  the  incoiping  "signal"  S  is  present  through 
the  change  of  N  to  N  appearing  in  the  antecedent. Fix 
throughout  the  discussion, in  the  antecedent  for  Q,an 
arbitrary  combination  of  possible  domain  values  for 
HYP^,ALG’^,DET+,  and  N+  and  hence  S,up  to  some  variabil- 

Consider  then  the  two  following  simplified  ex- 


ampl  es : 


Q  »  (alb)  V  (aje)  , 


(2.2) 


a  *  ship  target  possible  position  area  updated, 
b  *  track  history  1  is  the  assumed  assignment 
(possibly  in  error)  to  the  target, 
c  *  track  history  2  is  the  assumed  assignment 
(possibly  in  error)  to  the  target  . 

Or,  perhaps,  Q  represents  intelligence  informa¬ 
tion  to  be  fused  by  evaluating  the  truth  of  the  ex¬ 


pression 


where  now 

a  «  a(x)  =  enemy  will  move  up  about  x  troops  to¬ 
morrow;  x=0,50,100,150. 

b  •  b(y)  =  it  will  y  tomorrow;  y=be  clear,  snow, rain. 

c  =  c(z)  =  enemy  will  use  Pass  z  to  approach  us; 
z«I,II,III,IV. 

d  =  d(r,s)=  morale  of  enemy  node  17  is  at  level  r 
and  number  of  their  troops  left  is  s  ; 
r=very  low,  low,  medium, high,  very  high, 
s=0, 100 ,200, 300. 

e  *  e(w)  *  enemy  will  w  tomorrow;  w=surrender,not 
surrender. 

f  f(q)  =  enemy  overall  damage  level  is  q  ; 
q=0,l,2,..,10. 

Of  course,  if  the  antecedents  in  (2.2)  or  (2.3) 
were  all  the  same,  then  no  real  problem  would  arise, 
since  for  example  It  is  readily  justified  that  for  any 
choice  of  ALDP  -  certainly  for  PL  1  -  that 


Q  =  ((a|d)  &  (c|d))  V  (e|d) 

«  (((a&c)v^|d)  ,  (2.4) 


even  though  normally  one  does  not  talk  about  such 
meaeure-free  entities  (up -fco  now).  Indeed,  since  the  goal 
is  the  evaluation  of  Q,  for  PL,  choosing  a  probability 
measure  p  over  all  the  relevant  events,  one  would  us¬ 
ually  evaluate  Q  as  simply 


p(Q)  =  P(((a|d)  &  (cjd))  v  (e|d)) 
*  pff(a&c)ve) |d) 

=  p(((a8ic)ve))  &d)/p(d) , 


(2.5) 


etc.,  assuming  p(d)>0. 

But  the  point  of  the  above  examples  in  (2.2) 
and  (2.3)  is  that  the  antecedents  in  the  conditional 
forms  are  not  in  general  identical!  What  to  do? 

Contrary  to  popular  belief  [author's  note:  this 
author  and  his  colleague  Prof.  H.T.  Nguyen,  Math.  Dept. 
New  Mexico  State  University,  Las  Cruces,  have  undertak¬ 
en  an  extensive  informal  survey  of  the  probability 
community-  both  applied  and  theoretical -resulting  in 
the  following  conclusions  -  see  also  [7]  ];  there  is 
no  systematic  and  mathematically  sound  procedure  for 
computing  p(Q)  (or  Q,  for  that  matter)  in  either  (2.2) 
or  (2.3),  or  any  similar  problem! 


Indeed,  there  do  exist  "folk"  remedies  to  this 
situation  which  roughly  speaking  reduce  to  two  ap¬ 
proaches  as  presented  in  the  next  sections. 


2.2  Approach  A:  Identification  of  conditionijm 
with  material  implication. 


In  this  approach,  one  interprets  (a|b)  as  b»#a, 
for  any  a, b, events,  j 
(a  I b)  *  b=^  a  “ 


b'va  =  b’vraab) 


(2.6) 


is  assumed  to  be  valid  so  that  by  the  principle  of 
substitution  relative  to  equality,  for  any  (suitable) 
probability  measure  p,  assuming  the  Identity  for  p(b)>0 

p{(a|b))  .  p(a|b)  =  p(a&b)/p(b),  (2.7) 


one  has  immediately  from  (2.6) 


etc. 


p(a]b)  =  p(b=>a) 

=  p(b‘vi^&b))=l-p(b)+p(a&b) , 


(2.8) 


(2.9) 


Thus  using  this  approach,  (2.2)  becomes: 
p(Q)=p((b^a)  V  (c=>a))=p(b'vavc'va) 
=p((b&c)'va)*p((b&c)=»a  ) 

*p(a|b&c). 

However,  there  is  just  one  flaw  in  the  above 
reasoning:  One  cannot  make  the  identification  in  gener¬ 
al  in  the  left  hand  side  of  (2.6).  Indeed,  it  is  rather 
easy  to  show  (yet  surprisingly  few  are  aware  of  the 
inequality  below-  see  e.g.  the  discussion  in  [7],  sect. 
1.8): 


p(b^a)  ■ 


p(a 

p(a 


S:  p(aib 


b)  +  p(a' |b)-p(b’) 


!!• 


(2.10) 
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where  In  general  strict  inequality  holds  for  the  last 
two  Furthermore,  (2.6)  is  not  even  a  good  approxi- 
matibn^since  it  is  readily  verified  for  b  with  p(b) 
small  that  one  can  choose  a  and  b  such  that  p(b=>a)  is 
close  to  one„while  p(a|b)  Is  close  to  zero.  This  un¬ 
fortunate  situation  is  a  special  case  of  the  "Stalnaker 
Thesis"  problem  and  is  considered  in  detail  in  [6]  and 
[7],  section  1.  In  fact  the  following  apparently  not- 
well- known  result  has  been  shown: 

Theorem  2.1 .  (P.  Calabrese  [8],  section  1.2) 

Let  R  be  a  boolean  algebra  of  events  with  the 
usual  operations  discussed  previously.  Let  p  be  any 
non-degenerate  probability  measure  over  R,i.e. 
p:R  [0,1].  Then,  there  is  no  binary  boolean  function 
f:R2  R  such  that  for  all  aTF  e  R  ,  with  p(b)>0, 

p(a|b)  =  p(f{a,b)).  (2.11) 

I 

The  result  Is  further  extended  in  one  direction 
by  the  following  result: 

Theorem  2.2.  (  Goodman  &  Nguyen  [7],  Theorem  2,7) 

Let  R  be  any  finite  boolean  algebra  of  events. 
Then,  there  is  no  binary  function  f  of  any  kind,f :r2->r^ 
such  that  for  all  a,b  e  R,  with  p(b)>0,  (2.11)  holds. 

I 

However,  by  allowing  infinite  boolean  algebras 
one  can  force  a  form  of  conditioning  to  be  back  in  the 
original  boolean  algebra  (but  not  without  complication) 
as  Copeland  showed.  (For  a  critique  of  Copeland's  "im¬ 
plicative"  boolean  algebras,  see  [7],  section  1.8.) 

2.3  Approach  B:  Identification  of  conditional  events 
as  marginal  ones  with  common  joint  antecedents. 


Thus  one  is  lead  to  question  whether  any  remedy 
exists  for  this  situation;  Can  a  calculus  of  measure- 
free  conditional  events  be  developed  which  is  both 
compatible  with  ordinary  conditional  probability  eval¬ 
uations  and  is  also  unambiguous  and  feasible  to  Imple¬ 
ment, as  well  as  being  based  upon  sound,  non-ad  hoc 
mathematical  principles?  Certainly,  all  evaluation  of 
fused  data,  and  hence  evaluations  for  the  overall  C3 
problem,  must  depend,  in  effect,  on  the  outcome  of  this 
question. 

It  is  the  contention  here  that  the  answer  to  the 
above  question  is  definitely  in  the  affirmative.  In 
this  paper  in  section  3  an  outline  of  a  theory  for  de¬ 
veloping  such  a  calculus  of  operations  and  related 
issues  is  presented.  This  is  carried  out  for  not  only 
the  purpose  of  keeping  this  paper  as  self-contained 
as  possible,  but  also  because  of  the  desire  to  dis¬ 
seminate  these  novel  and  universally  applicable  re¬ 
sults  to  as  wide  an  audience  as  possible,  within  a 
short  time  period.  For  earlier  efforts  in  this  direct¬ 
ion,  see  [6J.  In  [7]C  in  the  process  of  being  submitted 
for  publication)  the  full  theory,  with  all  proofs,  is 
exhibited. 

One  consequence  of  the  calculus  of  conditional 
events  is  that  the  evaluations  for  Q  in  (2.2)  and  (2.3) 
become  rather  simple.  Thus,  it  will  be  shown  that 
(2.2)  yields 

Q  *  {(a&(bvc)|{a&(bvc))v(b&c))  (2.16) 

so  that 

p(Q)  =  p(a&(bvc))/p((a&(bvc))v{b&c)),{2.17) 

etc.,  differing  considerably  from  that  proposed  by 
Approaches  A  (see  (2.9))  or  B. 

For  (2.3) ,  one  obtains 


In  this  approach,  one  attempts  to  obtain  a  com¬ 
mon  joint  antecedent  event  for  all  of  the  conditional 
events  appearing  and  then  identify  each  as  a  marginal 
one  having  a  coirmon  joint  antecedent.  In  turn,  one 
precedes  to  evaluate  for  a  suitably  chosen  probability 
measure  analogous  to  that  in  (2,5).  This  is  best  shown 
through  an  application  to  (2.3): 

First  make  the  identifications 
(a|b)  with  (axdxflG)l  ^ 

(c|d)  with  bxcxfiG)  VjG  =  bxdxf.  (2.12) 

(e|f)  with  (bxdxe|G) J 

Then  (2,3)  becomes 

Q  =((axdxf|G)  &  (bxcxfjG))  v  (bxdxe|6) 

=  (((a&b  X  c&d  X  f)  v(bxdxe))|G)  ,  (2.13) 


res ul tin 
where 


Q  ==  (oi{B)  (2.18) 

g  in 

P(Q)  =  p(a)/p(B),  (2.19) 

a  ^  (a&b&c&d)  V  (e&f)  .  (2.20) 

6  =  a  V  (((a'&b)v(c'M)v(b&d))4f),  (2.21) 


differing  again  considerably  from  both  Approaches  A 
and  B  (see  (2.13)). 


In  all  of  the  above  computations,  one  need  not 
construct  joint  probability  measures  and  consider  car¬ 
tesian  products  of  events,  nor  is  the  procedure  ad  hoc 

(despite  the  oversimplified  appeal  of  Approach  A  -  but 
see  section  4 

3.  OUTLINE  OF  A  THEORY  OF  MEASURE-FREE  CONDITIONAL 
EVENTS 


resulting  in  the  evaluation 


3.1  Introduction . 


p(Q)  -  P^(a&b  X  c&d  X  f)  V  (bxdxe)  |  G)  ,  (2.14) 

which  can  be  further  reduced  using  the  definition  of 
conditional  probability  and  the  standard  calculus  of 
operations  for  PL. 


However,  the  main  drawback  to  this  approach  is 
that  initial  probability  measure  p:R [0,1]  must  be 
replaced  by  some  joint  probability  measure  p  over  the 
boolean  (or  sigma^  algebra  spanned  by  r3,  where 
p(- Jb) ,p(‘ •  |d) ,p(. . .  |f) :R [0,1  ]  are  conditional  prob¬ 
ability  measures  whose  joint  measure  is  Pq>so  that 

p(a |b)=p^{axdxf |G) ,p(c (d)=p  (bxcxf |G) ,p(e |f)=p(tKdxelC? 

°  °  ”(2.15) 

But  what  choice  of  p  to  make?  Should  it  be  based  on 
maximal  entropy  considerations,  etc.?  Long  calculations 
can  also  result  from  the  cartesian  product  forms. (For 
further  discussion,  see  [7],  sections  1.1,  1.5,  and 
10.4.  However,  for  a  tie-in  with  the  approach  presented 
in  this  paper.  Theorem  4.^  sWen  in  section  4  fs  of  use.) 

2-4  The  basic  problem  in  representing  conditional 
events  with  distinct  antecedents. 


As  shown  in  section  2,3  two  of  the  most  common 
approaches  to  the  handling  of  conditional  events  do 
not  lead  to  satisfactory  results,  from  either  a  math¬ 
ematical  v1ew)oint,as  in  Approach  A, or  an  unambiguous 
computationally  efficient  viewpoint, as  in  Approach  B. 


Following  the  basic  motivation  for  the  develop¬ 
ment  of  a  conditional  event  calculus  for  PL  in  section 
2,  this  section  presents  an  overview  of  the  basic  re¬ 
sults.  The  following  three  questions  are  addressed: 

(i)  What  meaning  can  be  attached  to  a  typical 
conditional  event  (a|b),  where  a  and  b  are  ordinary 
unconditional  events,  prior  to  evaluating  through  a 
specific  probability  measure  p  to  become  (for  p(b)>0) 

p((a|b))  =  p(a|b)  =  p(a&b)/p(b)  ?  (3.1) 

(ii)  How  shall  operations-  in  particular,  the 
usual  boolean  operations  &,v,()'  and  relations  such  as 
^  -  be  extended  from  unconditional  events  to  condition¬ 
al  ones  and  what  properties  do  they  possess? 

(iii)  Can  such  operations  and  relations  as  in  (ii) 
be  characterized  for  uniqueness,  etc.  ? 

For  a  history  of  previous  attempts  at  develop¬ 
ing  a  theory  of  measure-free  conditioning,  see  [7], 
sections  1.8  and  1.9.  Among  the  predecessors  of  this 
effort,  Schay  [9]  was  among  the  first  to  attempt  such 
a  task,  but  used  an  ad  hoc  procedure  in  addressing 
question  (ii),  although  a  somewhat  complicated  charact¬ 
erization  was  developed  relative  to  (iii).  Later, 
Calabrese,  completely  independent  of  Schay,  produced 
also  an  algebra  of  conditional  events  and  operations. 
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the  latter  also  from  an  ad  hoc  approach  following  cert¬ 
ain  analogies  with  material  implication  [8]. 

3.2  Development  of  measure- free  conditional  events . 

From  now  on,  without  further  explanation,  the 
symbol  R  refers  to  an  arbitrary  fixed  boolean  algebra 
with  all  of  the  usual  operations  and  relations  explain¬ 
ed  previously  (see  section  1.2).  The  partial  order 
s  over  (corresponding  to  the  subset  relation  c 
among  sets  corresponding  to  events  or  propositions 
as  considered  here)  is  defined  as  usual  as 

asb  iff  a  =  a&b  iff  b  =  avb,  (3.2) 

for  any  (unconditional)  events  a,b  e  R.  Note  also,  0 
denotes  the  null  event  and  fi  the  universal  (unity) event. 

Define  tf,S}  to  be  any  candidate  olaee  of  con¬ 
ditional  events  extending  R  iff  .f  is  some  function 
f:R^  5  for  some  space  S,such  that  for  all  a,b,c  e 
R,  and  boolean  operations  ★  over  r;  ,etc,  are  oper- 


ations 

over  (assumed  )  boolean  algebra  of  events 

(3.3) 

where 

^  f(*.b)(R)  =  range(f(*,b)) 

S  *  U  s  . 
b  e  R 

(3.4) 

f(a,b)*j^f(c,b)  »  f(a*c,b). 

(3.5) 

and 

f(a,b)  *  f(a&b,b)  ,  f(a,fi)  =  a  . 

(3.6) 

For  any  candidate  class  (f,S),  one  can  interpret 
f(a,b)  as  a  conditioned  upon  b  wrt  f  or  symbolically 
as  (alb).,  where  a  Is  the  consequent  and  b  the  ante¬ 
cedent.  * 

Theorem  3.1,  ( [7],  Theorem  2.1) 

(i)  (nat,R)  is  a  candidate  class  of  conditional 
events  extending  R,  called  the  canonical  class  of 
conditional  events  extending  R,  where  in  C3»4> 

where  for  each  a,b  e  R» 

(alb)  i  (a|b)^jj  =(m*)v{a&b) 

=  {(x&b')v(a&b):  XeR}  (3.8) 

is  the  principal  ideal  coset  (R&b'  being  the  princi¬ 
pal  ideal)  generated  by  b'  with  residue  a&b,  making 
Rj^  a  boolean  quotient  algebra  with  the  usual  coset 

operations  corresponding  to  ordinary  boolean  oper¬ 
ations  *  over  R  (thus,  *  =  &,v ,() ' ,etc. ) . 

r4  Ur.*  {(a|b):  a,b  e  R>  (3.9) 

beR 

and  for  any  a,b  e  R, 

nat(a,b)  =  (ajb).  (3.10) 

(ii)  For  each  a,b  e  R,  noting  (ajb)  c  r  (alb)  is 

the  inverse  of  the  conjunction  operation  (•&b):R’>R  at 
a&b,  i.e. ,  ^ 

(a|b)  *  {y:  yeR  and  y&b  *  a&b}.  (3.11) 

Equivalently,  (a|b)  is  the  largest  subclass  of  R  sat¬ 
isfying  the  relation 

(a|b)&b  =  a&b  (3.12) 

(Hi)  For  ^  candidate  class  (f,S)  of  conditional 
events  extending  R,  and  each  beR, 

S  (&  ,v.  ,()'b)fiS  R/f“H- .b)  (boolean  quot.alg.) 

the  symbol  denoting  an  isomorphism,  so  that  in  the 
above  local  sense  for  each  b,  the  canonical  class  of 
conditional  events  is  the  smallest  possible  candidate 
class  of  conditional  events  extending  R.  ^ 

Note  also  that  for  any  a,b,c,d  e  R» 

(alb)  =  (c|d)  iff  a&b  =  c&d  and  b  =  d  ,  (3.14) 

but  unlike  the  classical  case  where  b=d,  (db)  is  not 
necessarily  identical  nor  disjoint  from  (cld).  (See 
[7],  Theorem  2.11)  Note  also  the  relations 


R  c  R  £  p(R),  (3.15) 

where  p(r)  denotes  the  power  class,  or  class  of  all 
subsets  of  elements  ,  of  R. 

Next,  call  any  pair  (f,S)  (  not  necessarily 
a  priori  a  candidate  class  of  conditional  events) 
where  f :R2->S  is  surjective  (i.e.,  onto)  a  probability- 
compatible  pair  with  respect  to  R  iff,  by  definition, 

R  c  5  “  u  Su  (3.16) 

beR  ^ 

where  now  some  boolean  algebra,  for  each  beR, 

and  where  for  each  probability  measure  p:R->[0,l]  and 
each  beR  with  p(b)>0,  p  can  be  extended  to  (using  the 
same  symbol)  p:i“>-[0,l]  such  that  for  all  aeR, 

p(f(a,b))  =  p(a|b).  (3.17) 

Theorem  3.2#  ([7],  Theorems  2. 5,2. 9) 

Let  (f,S)  be  a  probability-compatible  pair 
with  respect  R.  Then: 

(1)  (f*S)  is  also  a  candidate  class  of  conditional 

events  extending  R. 

(ii)  (nat,Rl  is  a  probability-compatible  pair. 

(iii)  By  suitable  restriction  of  f:R2  ^  5  to  non¬ 

trivial  ,i  .e.  ,  non-zero  or  non-unity  probability-valued 
conditional  event  pairs,  tiKen  S  f  becomes  bijective 
and  ^  in  (3.13)  can  be  replaced  by  a  global  isomor¬ 
phism  5^  not  dependent  upon  b,  showing^that  the  canon¬ 
ical  class  of  conditional  events  (nat,R)  is  the  uni¬ 
versally  minimal  (wot,  subclass  inclusion)  probability- 
compatible  candidate  class  of  conditional  events  ex¬ 
tending  R.  ^ 

Next,  a  basic  logical  justification  for  choos¬ 
ing  the  canonical  conditional  events  is  given.  First, 
for  any  a,b  e  R,  define  the  class  of  all  di-relative 
deducts  o/  b  as 

4a lb)-  =  {x:  xeR  and  there  exists  reR  with  rib 

and  x&r  =  a&r  }  (3.18) 

(Calabrese  [8],  motivated  by  classical  logic  deduction , 
proposed  this  definition  previously  for  conditioning 
a  by  b.) 

Theorem  3,3.  ([7],  Theorem  1.3) 

For  all  a,b  e  R, 

4a|b).=  (a|b),  (3.19) 

i.e.,  R  coincides  with  the  class  of  all  relative  de¬ 
ducts! 


3 . 3  Calculus  of  operations  on  conditional  even ts . 

With  the  role  of  the  canonical  class  of  condit¬ 
ional  events  firmly  established  in  section  3.2,  consid¬ 
er  now  the  choice  of  operations  upon  them  extending 
the  usual  boolean  operations  acting  upon  unconditional 
events.  Throughout  all  of  real  analysis  and  topology, 
a  universal  way  of  extending  a  given  "poinf'-valued 
function  to  a  '‘set‘'-valued  one  is  simply  through  the 
natural  extension  or , equivalently ,  called  the  compon¬ 
ent-wise  class  or  image  extension.  In  particular,  let 
g:R^  -►  R  be  any  binary  operation  (boolean  or  otherwise) 
upon  R  (unary  and  more  generally,  n-ary  operations  can 
be  treated  similarly).  Then  the  natural  extension  (also 
denoted  by  the  same  symbol  for  a)  from  g  over  R^  to 
g:P(R)2  -^^P^R),  i.e.,  over  p(R)%is  determined  by, 

g(A,B)  =  {g(a,b):  aeA,  beB},  (3.20) 
for  any  A,B  c  R  ,i.e.,  A,B  e  p(R), 

Thus,  recalling  the  comments  in  Theorem  3.1 (ii) 
and  (3.15),  it  is  basic  to  inquire  what  forms  the 
natural  extensions  take  for  the  ordinary^boolean  oper¬ 
ations  relative  to  R  when  restricted  to  R.  Note  that 
for  the  binary  operations  &,v:R2  -►  R  »  by  their  com¬ 
mutativity  and  associativity:  they  extend  recursively 
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and  unambiguously  to  &,v:R^  -►  R,  where,  by  convention, 
for  n*=l ,  &  =  V  *  identity  function.  (The  unary  negation 
or  complement  operator  ()':R  -v  R  remains  as  Is.) 

Denote  the  natural  extensions  of  &,v,()'  by  the  same 
symbols.  Thus,  here  the  behavior  of  the  natural  exten¬ 
sions  of  &,v;R  R  to  the  restrictions  &,v:^"  P(R) 

and  that  of  ()’:R  R  to  the  restriction  ()':R  p(r) 
are  sought. 

Theorem  3.4^  ([7],  Corollaries  3. 3,3.4) 


For  any  positive  Integer  n  and  any  ax»b-  e  R, 
j=l,..,n;  J  J 

(1)  The  common  antecedent  case*  noting  Rj^  c  R. 

(a|b)^(a|b)\  »  (a'|b)  «  (a'&b|b),  (3.21) 

*  (ail**}  •  ^  *  ajb)«(  *  a,4b|b),  (3.22) 

j»l  ^  j.?  J  J  ^ 

for  *  =  &,v. 

Thus,  relative  to  any  given  boolean  quotient 
algebra  of  the  form  R.>all  coset  operations  and  cor¬ 
responding  natural  extensions  of  the  original  opera¬ 
tions  relative  to  R  coincide.  It  follows  that  If 
g:R^  -►  R  is  any  (compound)  boolean  operation,  then 

g((a,lb^)....(ajbj)=g^(a^|b,)..,,(ajbj) 


•  (g(a^,...a^)|b). 

(11)  Distinct  antecedents;ln  general, case. 

&(a  Jb  )  =  (  &  aJr)  =  (  &(a.&b.)|r), 
j-1  J  J  j=1  '•  j=l  J  ^ 


(3.23) 

(3.24) 


where 


and 


n 


n 


&  b.  V  v(a;&b,)=  &(a.&b.)  v  v(a’ &b.) 

J  j=H  J  j«i  J  j=i  ^ 

(3.25) 


where 


II  II 

vla,|b.)  =  (  V  aJg)  ' 

j=1  J  J 

d  "  " 

*  v(a.&b  )  V  &b^ 

j-1  J  j=T’ 


(  D(a.&b,)|q).  (3.26) 

j=l  J  J 


=  !!(a.&b  ) 


n 

V  &(a’,&b,). 

j=l  J  ^ 

(3.27) 


Combining  (3.24)  and  (3.26),  leads  to  the  fol¬ 
lowing  corollary: 


Corollary  3.1  (New  result.) 

Noting  that  any  (compound)  boolean  function  of 
multiple  arguments  can  always  be  put  into  an  equival¬ 
ent  form  consisting  of  a  disjunction  of  conjunctions, 
let  £  R.i*l ...  ...  ,n,  where  some  of  these 

events  may  be  0  or  fi  .  Then 
m  n 


V  i 
1*1  j«l 

hji 

=  (  a 

1  8  ). 

(3.28) 

where 

d 

a  =  V 

i=l 

&( 

I  j-1 

• 

8  =  0  +  Y  , 

(3.29) 

n 

m 

a 

Y  = 

& 

j=l 

v(a'. . 
1=1 

*•>13)  • 

(3.30) 

noting 

that  a 

and 

Y  are 

disjoint,  i.e.. 

a  & 

Y  =  0 

» 

(3.31) 

so  that  for  any  probability  measure  p:R  -►  [0,1], 
m  n 


P( 


V  & 

1=1  j=l 


(aij|bij))'p(a|6)=p(o)/p(B)=1/0-Kp(Y)/p(a))), 

(3.32) 

■ 


Theerwi  3^6.  Some  special  cases  ([7],  section  4.2). 

For  any  a,b,c,d,a^  e  R>  1°1,..,n: 

(a|n)«a,  (a|e)*R,  (*|b)*(b*|b)«RSb*«{x&b' :  xeR},(3.33) 
(fi|b)-(b|b)-Rvb'-{xvb’;  xeR},  (3.34) 

(a|b)&(c|d)=(a&b&c&d|(a’«,b)v(c'&d)v(b&d)) 


=(a&b&c&d|(a'&b)v(c'&d)v(a&b»c»d)),(3.35) 
(a|b)v(c  |d)=((a8b)v(c8d) | (a&b)v(c&d)v(b8d)) 

=((a&b)v(c&d) I (a&b)v(c&d)v(a'&b8c'&d)) , 

(3.36) 

(a|b)=(a&b|b)=(b=>a  |  b)«(b’^a)&(b  |b)«((bia')va)&(b|b)  , 

(3.37) 

(a|b)Sb=a&b  ,  (a|b)vb=(blb).  (3.38) 

(a|b)v(bla)=(a&b|a&b).  (a|b)&(b|a)=(a&b|avb).  (3.39) 

(a|b)v(c|b')=((a&b)v(c&b')|(a&b)v(c&b')),(3.40) 
(a|b)&(c|b')=(e|(a'&b)v(c'&b')),  (3.41) 

(c  d)=>(a|b)  =(c-.’a  |(c'&d)v(a&b)v(b&d)),  (3.42) 

(c  d)4^  (a|b)  =  (c«*a|b&d),  (3.43) 

chaining:!  =(a|bSc)&(blc).  (3.44) 

l(a&b&c|d)  =(a|b8c8d)8(b^c&d)&(c|d),  (3.45) 

Bayes*  Thiti:  (ajb)  =((b|a.  )&a  ((b|aj)&a^))  ,(3.46) 

provided  that  n 

b  ^  V  a.  ,  (3.47) 

where  above^materlal  implication  and  material  double 
impllcatlon^as  extended  from  R  to  R^are  defined  formal¬ 
ly  the  same  as  in  the  unconditional  case  J 

(c|d)  »»(a|b)  =  (c|d)*v(a|b),  (3.48) 

(c|d)  0(a|b)  =((c|d)=>{a|  b))&((a|b)^(c|d))j3.49) 

B 


Recalling  the  partial  order  s  for  R  (see  (3.2))^ 
define  and  extend  this  relation  to  R  by  letting,  for  * 
all  a,b,c,d  e  R, 

(a|b)  ^  (c|d)  m  (alb)  =  (a|b)&(c|d).  (3.50) 

Theorem  3.7.  ([7],  Theorems  4. 1,4. 7) 

For  all  a,b,c,d  e  R: 

r-2 

(I)  Characterization  of  ^  over  R  : 

(a|b)  ^  (c|d)  Iff  (cld)  =  (alb)v(c|d) 

rff{a&b)^  (c&d)  and(b=»a)^(d=>c) 
m (aab)^(c&d)  and  (c‘&d)^(a ‘&b) 
iff  (c|d)‘  (a|b)'.  (3.51) 

«w2 

(II)  s  over  R  Is  not  only  a  partial  order  (reflex¬ 
ive,  transitive,  anti -symmetric) ,  but  also  a  meet  (&) 
and  jo1n(v)  lattice  with  all  of  the  usual  operation-^ 
preserving  properties.  Note  the  relation,  compatible 
with  (2.10)  and  (3.8),  showing  that  b=»a  and  a&b  are 
the  largest  and  smallest  elements  in  (a|b),  respect¬ 
ively,  relative  to  partial  order  s  : 

a&b  s  (a|b)  £  b=>a  .  (3.52) 

(ill)  R(&,v,()‘;^)  is  a  distributive  lattice  which  is 
also  an  algebraic  semi -ring  with  zero  element  0  and 
unity  element  n,  and  is,  further,  idempotent,  absorb¬ 
ing,  and  demorgan,  among  other  properties. 

I 

3.4  Justifications  for  choice  of  extensions  of 
0  per  a  1 1  o  ns~frbm~  R~~t  o  R. 

In  addition  to  the  large  number  of  desirable 
properties  for  the  naturally  extended  boolean  opera¬ 
tions  upon  ^  given  In  section  3.3,  characterizations 
can  also  be  established. 

Recall inq  the  maximal ity  property  of  b=»a  with 
respect  to  fa|b)  ((3,52)),  define  a  corresponding 
mapping  -j-  R,  where  for  any  a,b  e  R, 

♦  ((a|b))  =  .  (3.53) 

Theorem  3.8.  (£7],  Theorem  10.1,  Remark  10.1) 

(I)  <ji  Is  a  surjective  &,v-homomorph1sm  wUh  respect 
to  the  natural  extensions  of  &,v  from  R  to  R. 

(II)  Let  &,v:R  R  be  any  possible  extensions  of 
&,v:R^  -►  R,  respectively,  such  that  there  exist 
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functions  41^  .Tj'2  where  for  all  a,b,c,d  e  R, 

a&b  V  c&d  ^  ilro(a»b,c,d),  a&b&c&d  ^  4i-(a,b,c,d) 

2  1  (3_54) 

and  ^  . 

(a|b}&(c|d)  =  (a&b&c&dlif;^(a,b,c.d)),  (3.55) 

{a|b)v(cld)  =  ((a&b)v(c&d)  |\p2U.b,c,d)),  (3.56) 

noting  the  essential  necessity  of  consequences  matching 
the  corresponding  ones  for  the  natural  extensions. 

Then  it  follows  that  relative  to 

&  =  &  ,  V  =  V  .  (3.57) 

(iii)  Compatible  with  Theorems  2.2,  3. 1.^3. 2,  there 
is  no  full  (i.e,  &,v,()-) homomorphism  p:R  -►  R  with 
respect  to  the  natural  extensions  of  &,v,()'  from  R 
to  R  .  I 


p(a|b)-l,  t3.66) 

for  all  probability  measures  p:R  ^  [0,1],  with  p(b)>0. 

From  [7],  sections  9.4,  9.5,  2.3,  the  following 
concepts  and  results  hold,  for  any  a,b,c,d  e  R; 

(3.67) 

f*-  (a|b)  iff  b^a  i_^  b=»a  =  1  iff  (a|b)=(b|b), 

CPL 

"'h-  (alb)  and  |=  (bla))vff  a=b  iff  aOb  =1 

iff  (a|b)  =  (a|a)=(b|b),  (3.68) 
(III)  Define  (cld)  tautologically  implies  (alb)  i ff 

(3.69) 

Then  |_  ((c|d)=>(a|b))  iff  b&c&d  ^  a  iff  c&d  <  b=^a. 

CPL  (3.70) 


Theorem  3.9.  Partial  converse  of  Theorem  3.8. 

([7], Theorem  10.2,  Corollary  10. 
Remark  10.2) 


1. 


Let  &,^:R^  -►  R  be  any  possible^extensions  of 
the  corresponding  coset  operations  Rj^>  sim¬ 

ultaneously  an^  consistently  for  all  b  e  R.  (Hence, 
necessarily  ,  &,v  extend  &,v  relative  to  R.) 

Suppose  also  that  &,v  obey  not  only  closure, 
but  are  also  commutative  and  associative  over  k  with 
0  and  fi  playing  the  usual  roles  of  zero  and  unity  el¬ 
ements,  respectively.  Suppose  also  there  exists 


,ii>2:R‘*  -►  R  such  that  for  all  a,b,c,d  e  R, 

(alb)&(cld)  =  (a&b&c&d (a, b,c,d))  ,  (3.58) 

(alb)v(cld)  =  ((a&b)v(c&d)  »h»c,d)) ,  (3.59) 

(alb)&b  =  a&b  ,  (3.60) 

and  w  / «  \ 

(  (a|b)  &c  e  R  )  implies  csb.  (3.61) 


Then 

(i)  For  all  a,b,c,d  e  R, 

(a*&b)v(c’&d)v(b&d)  ^  i|»^(a,b,c,d),  (3.62) 

(a&b)v(c&d)v(b&d)  ^ 

showing  that  the  natural  extensions  of  &,v  are  maximal, 
(a|b)&(c|d)  ^  (alb)&(c|d).  (3.64) 


(alb)v(c|d)  i  (a|b)v(c|d).  (3.65) 

(i1)  Result  (1)  shows  that  in  contradistinction  to 
the  antecedent-only  dependent  operations  of  Schay  19] 
and  Calabrese  [8],  there  are  boolean  functions 
^pi|>2  as  above, but  now  such  that  for  all  a,b,c,d  e  R, 
ii;j(a,b,c,d)«if.j(b,d)  only,  j=l  ,2^such  that  the  corres¬ 
ponding  operations  &  and  v  satisfy  the  hypotheses  of 
this  theorem.  ■ 


3.5  Additional  properties  ^  R. 

Finally,  brief  mention  should  be  made  of  other 
pertinent  properties  of  the  conditional  extension  of 
a  given  boolean  algebra  of  events. 

(1)  Stone's  Representation  Theorem  -showing  an 
order-preserving  isomorphism  always  exists  between  a 
given  boolean  algebra  R  of  events  or  propositions 
and  a  corresponding  boolean  algebra  of  subsets  of 
some  set -can  be  extended  quite  readily  to  R  ([7J, 
Theorem  10.3). 


(11)  The  usual  Hilbert-Ackermann  axioms  involving 
material  implication, relative  to  any  R,  wher  R  is  re¬ 
placed  by  R,  ordinary  substitution,  and  a  modified 
form  of  modus  ponens  used  to  deduce  theorems  from  pre¬ 
vious  theorems  and  axioms,  together  with  the  natural 
extensions  of  all  boolean  operations  from  R  to  R, 
forms  a  sound  and  compl ete  logic,  I.e.,  all  theorems 
are  tautologies  and  vice-versa,  Call*thls  conditional 
probability  logic  (CPL),  extending  ordinary  probabili¬ 
ty  logic  (PL).  (See  [7].  Corollary  9-1  Here, -for .any 
a,b  e  R,  (a|b)  is  a  tautology  /or  CPL,  written^  (alo) , 

iff,  by  definition. 


fIV)  Define  (c|d)  tautologically  is  equivalent  to 

CPL 

Then  ^  (a|b))  b&d  s  (aSb«^cSd).  (3.72) 

CPL 

(cld)i(a|b)  implies  H  ((cld)"»(a|b)),  (3.73) 

CPL 

(c|d)=(a|b)  implies  ((c|d)0(a|b)),  (3.74) 

CPL 

(VII)  Sufficient  conditions  for  modus  ponens  analogue: 

(c!d)^(a|b)  and  h  (^Id)  implies  (= 

CPL  CPL 

(VIII)  Characterization  of  modus  ponens  analogue: 

(  H((c |d)=>(a|  b))  and  (c|d)  ) 

CPL  CPL  (3,76) 

iff  (  [*=  (a&c|b&d)  and  f=(c|d)X 
CPL  CPL 

(IX)  h  ((a|b)^(c|d))  implies 

CPL  .  X  (3.77) 

H  ((alb)=^(c|d))  and  ((cld)=»(al  b))  ), 

CPL  CPL 


(X)  Next,  define  [z\(\)  semantically  [or  uniformly  in 
probability)  implies  (a|b),  written  (c|d)  (a|b)  , 

CPL 

~  p(c|d)  ^  p(a|b),  (3.78) 

for  all  probability  measures  p:R  [0,1]  with  p(b), 
p(d)>0. 

Also,  define  any  (a|b)  to  be  a  contradictvon 
for  CPL,  written  =/=  (a|b)  ,  vff  for  all  probability 
CPL 

measures  p:R  [0,1],  with  p(b)>0, 

p(alb)  *  0.  (3.79) 


It  readily  follows  that 

M=(alb)  iff  a&b  =  0  vff  f==(a|b)'.  (3.80) 

CPL  CPL 


With  a  bit  more  difficulty,  one  can  show 


(cld)  -J-  (a|b)  vff(  h  (a|b)  or^  (c|d)  or  (c|d)<(a|b))^ 
CPL  CPL  CPL 

(XI)  Call  (alb)  and  (c|d)  sermntically  [or  uniformly 
in  probability)  equivalent^  written  (c|d)^(a|b),  2II 

p(a|b)  =  p(cld),  (3.82) 

for  all  probability  measures  p:R  ->•  [0,1]  with  p(b), 
p(d)>0. 

It  follows  that 

(cld)<^(alb)  iff((h(alb)  and  [=(c|d))  or 

CPL  -  CPL  CPL^ 

( ff  (alb)  and  ff  (c|d))  or 

CPL  CPL 

(a|b)  =  (cld)).  (3.83) 

(XII)  Finally,  mention  can  be  made  of  a  weaker  form 
of  semantic  implication  by  restricting  probabilities 
in  the  above  definitions  to  subclasses  or  even  to  a 
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single  probability  measure. 

(Hi)  Another  area  of  basic  interest  concerns  higher 
order  conditioning.  Thus, if  beginning  with  uncondition¬ 
al  events  say  a,b,c,d  e  R  ,  it  is  meaningful  to  consid¬ 
er  the  conditional  events  (a|b),  (c|d),  then  why  should 
one  not  consider, in  turn>the  second  order  conditional 
event  ((a|b) | (c|d))  ?  (This  issue  is  addressed  in  [7], 
section  5  in  some  detail.)  Basically,  the  appropriate 
definition  for  the  above  expression  is  the  formal  high¬ 
er  order  analogue  of  (3.11).  Thus 

((a|b)|(c|d))  -  {(x|y):{x|y)eR,(x|y)&(c|d)-(a|b)8i[c|dn 

One  major  result  ([71,  Theorem  5.5)  concems 
the  class  union  mapping  a:P{p(R))  -»-p{R),  where  for 

anycJ(cp(R),  ^ 

a{CS)  {x:  xeAedJ  ,  for  any  AGtP}.(3.85) 

Noting  that  u  is  a  surjective  homomorphism  relative  to 
all  natural  extensions  of  operations  from  p(R)  to 
P(p(R)),  not  just  boolean  operations,  and  noting  that 
all  boolean  operations  over  ^  {£P(r))  can  be  extended 
to  p(r)  (cp(p(r)))  -In  the  natural  sense,  though  unlike 
the  first  order  case,  R,  closure  problems  arise  for 
the  higher  order  case,  the  following  reduction  holds: 

a(((a|b)|(c|d)))  =  {alb&((a'&d‘ )v(c&d))  ,  (3.86) 

for  all  a,b,c,d  e  R.  In  particular. 

a(({alb)|c)  =  a{((a|b)|(c|b)))  =  (a|b&c),  (3.87) 

Thus,  through  the  above  relations,  in  a  sense, 
one  need  never  consider  higher  order  conditioning. 

(iv)  One  can  establish  optimal  approximations  of 
arbitrary  subclasses  of  R  through  conditional  events, 
i.e. ,  if^Ac  R  ,  the  heBt  upper  approximation  of  A 
through  R  satisfies  the  relation  ([7],  section  6) 

A  c  cond(A)  ■  fl  {(aib):  A  c  {a|b)  e  R  }  £  r,  (3.88) 
noting  trivially  that  for  any  (a  b)  e  R, 

(a|b)  «  cond((a|b)).  (3.89) 

The  chief  results  include  the  following  (see 
[7j,  Theorems  6. 1,6. 2,  and  Corollary  6.3): 

(I)  If  now  R  is  a  compl ete  boolean  algebra,  then 
for  any  A  c  r, 

cond(A)  =  (&(A)|&(A)  v  (v(A))')  (e  R) ,  (3.90) 

where 

S(A)  =  &a  ,  v(A)  ^  va  (3.91) 

aeA  aeA 

are  the(possibly  non-finitelextended  definitions  of 
&  and  V  due  to  the  completeness  of  R. 

(II)  Let  R  =  B,  the  class  of  all  borel  subsets  of 
fi,  the  real  (or  one-dimensional  euclidean)  line.  Let 
gcK  ^  be  any  function.  Denote  the  natural  extension 
of  g  to  P(fi)  restricted  to  B  (s.P(b))  as  simply 
grB*"-)- B  (assuming  g  is  sufficiently  measurable).  In 
turn,  denote  the  natural  extension  to  p(b)  restricted 
to  B,  the  conditional  extension  of  B,  as  simply 

g:B -»'P(b). Then  for  all  (borel  sets)  a.,b  e  B,  with 
a.  c  b^. ,  1=1,,, m,  noting  as  ordinary  setJ,c  replaces 

n  replaces  &,  u  replaces  v,  etc., 

cond(g((a,|b,)...,(a^|b^))).  (a|6).  (3.92) 

Where  d  ,  ^  v 

a  “  g(a^ ,a^),  (3.93) 

6  *  a  u  (g(b^=>ap..,bj^=>ajj^))'  ,  (3.94) 

where  here  b^.=»a^  *  b'.  u  a^  . 

If  g  is  commutative  and  associative,  then  so  is 
cond(g)  :B  B  . 

The  above  development  is  particularly  useful  in 
determining  the  optimal  approximations  for  naturally 
extended  arithmetic  operations,  since  these,  unlike 
the  boolean  operations,  in  general  do  not  possess 
closure  properties  over  B.  These  results  lead,  in  se¬ 
quence,  to  the  development  of  random  conditional  events 


and  related  ideas.  (See,  in  particular,  [7],  sections 
7.3  and  10.4.) 

4.  PROBABILITY  EVALUATIONS  AND  APPLICATIONS 

Following  the  brief  overview  of  the  role  of 
data  fusion  in  developing  a  generic  model  for  typical 
C3  processes  in  section  1  and  the  motivations  and 
mathematical  and  computational  structures  for  coi'sdit- 
ional  events,  this  section  presents  the  fundamental 
links  for  probability  evaluation  and  application  to 
combining  of  evidence  and  data  fusion. 

First  note  the  following  theorem  : 

Theorem  4.1 .  ([7],  Theorems  1.5,  7.1,  Remark  1.3) 

Let  p:R  [0,1]  be  a  given  probability  measure. 
Then : 

(1)  With  slightly  additional  conditions  placed  upon 
p  (non-atomicity),  the  only  possible  extension  of  p 
to  first  p:R^  t0»l]  >  for  each  beR,  and  then  to 

p:R  [0,1],  where  p((a|b))  is  some  fixed  function 
(not  dependent  upon  any  given  a,b  e  R)  of  p(a&b)  and 
p(b)  such  that  p((- 1  b))  :R.->  [0,1]  is  a  probability 
measure  is 

p((a|b))  =  p(a|b)  =  p(a&b)/p(b),  (4.1) 

i.e.,  ordinary  conditional  probability  must  be  assign¬ 
ed  to  conditional  events. 

(ii)  The  extension  p:K-»'[0,l]  is  monotone  inoreasingt 
i.e.,  for  any  a,b,c,d  e  R, 

(a|b)  ^  (c|d)  implies  p(a|b)  ^  p(c|d).  (4.2) 

8y  using  the  demorgan  property  among  others 
(see  Theorem  3.7(iii)  and  Corollary  3.1  ),  all  com¬ 

putations  for  probabilities  of  (compound)  boolean 
functions  of  conditional  events  can  be  reduced  to  com¬ 
puting  probabilities  of  only  conjunctions  of  condition¬ 
al  events.  With  this  in  mind, the  following  result 
shows  that  the  measure-free  conditional  event  approach 
presented  here  in  conjunction  with  probability  evalu¬ 
ations  can, in  a  sense, be  identified  with  a  modified 
form  of  Approach  B  given  in  section  2.3,  loith  the 
joint  prohability  measure  p  ,  in  effect,  determined 
through  conditional  event  conjunction  and  initial 
probc^ility  measure  p  over  R: 

Theorem  4.2  (  [7],  section  10.4,  issue  (x)) 

Let  p:R  ^  [0,1]  be  a  given  probability  measure, 
let  b,d  £  R  be  arbitrary  with  p(b),p(d)>0.  For  each 
s,t  e  B,  consider  the  infinite  left  rays 

83  ^  (-,s]  ,  i  {-».t]  .  (4.3) 

Then,  p((ag |b)&(c^|d) ) ,as  a  function  of  (s,t), 

is  a  legitimate  cumulative  probability  distribution 
function  over  with  p( (a^ |b)&{dld}) ,as  a  function  of 

s,  and  p((c^|d)&(b|b)) ,  as  a  function  of  t,  being 

marginal  cumulative  probability  distribution  functions 
over  B. 


BASIC  PRIISCIPLES  FOR  EVALU- 
ATIM6  PROBABILITIES  OF 

COMDITIOPIED  INFORflATIOM 


(1) .  Determine  whether  or  not  the  evidence  has  truly 
differing  antecedents. 

(2) .  If  the  antecedents  of  the  information  are  identical, 
then  apply  the  usual  calculus  of  relations  for  PL.  For 
example,  suppose  the  same  source,  sensor  or  human,  on 
two  different  occasions  produces  an  estimate  of  target 
location  and  it  Is  desired  to  obtain  the  probability 
of  the  disjunction,  since  if  the  resulting  probability 
is  sufficiently  low,  no  further  investigation  will  be 
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carried  out.  If  the  source  error,  though  possibly  In¬ 
dependent,  Is  relatively  small,  then  both  target  est¬ 
imates  can  be  considered  unconditioned  information. 

Or,  perhaps,  by  sheer  coincidence,  the  same  error 
causing  mechanism  is  present  and  behaves  the  same  way 
in  producing  the  two  estimates,  which  themselves  can 
differ  greatly  in  terms  of  random  behavior  and  prob¬ 
ability  characteristics.  Or,  further,  where  Approach 
B  is  valid:  the  actual  joint  probabilities  of  the 
source  random  mechanism  are  known,  one  can  then  re¬ 
duce  all  computations  to  that  involving  only  PL,  be¬ 
cause  of  the  common  joint  antecedent. 

(3).  But,  in  general,  as  in  the  simplified  examples 
in  section  2.1,  the  antecedents  of  arriving  condition¬ 
ed  information  arise  from  widely  varying  sources,  of 
which  little  or  nothing  is  known  concerning  joint 
probability  distributions,  yet  relatively  much  is 
known  concerning  the  individual  (consequent)  event 
probabilities. 

fill  Isolate  all  the  relative  unconditional  events 
a,b,c,d,e,f,.,  making  up  the  forms  of  the  individual 
conditional  probabilities  p|a|b),  p^c|d),  i^elf),.. 

$1  Temporarily  ignoring  the  probabilities,  deter¬ 
mine  just  what  coRipound  boolean  operation  is  desired. 
Perhaps  one  simply  is  seeking  to  obtain,  in  some  way, 
the''joint"  probability  of  (ajb),  (c|d),  (e|f),i.e, 
in  actuality,  the  probability  of  the  conjunction 
(a|b)&(c|d)&(elf)  ,  or  any  other  combination  as  in 
the  examples  previously  mentioned.  Carry  out  the 
measure- free  computations,  based  upon  the  boolean 
function  desired,  in  accordance  with  the  calculus  of 
conditional  event  operations  as  given  in  section  3.3. 
Note  that  ary  such  operation  reduces the  collection 
of,  possibly  many, conditional  events  to  a  single  c^- 
ditional  events  say  (a|3),  where  now  a, 3  are  known 
boolean  functions  of  the  . input  events  a,b,c,d>^».f • 

€)l  Approximate  by  a  single  probability  measure  p, 
say,  in  place  of  all  the  differently  arising  probabil¬ 
ities  in  step  4,  so  that,  in  a  consistent  sense, 

p(a&b)=p^(a&b),  p(c&d)=p^(c&d) ,  p(e&f)=P3(e&f) ,  .. 

Obviously,  if  the  very  same  event,  say  b  is  assigned 
two  distinct  probabilities  from  two  sources,  say 
p. (b)  and  p.(b),  some  concensus  must  be  determined 
before  a  filial  common  assignment-  by  perhans  use  of 
least  squares,  maximum  likelihood,  or  maximal  entropy 
techniques. 

In  lieu  of  stepfA).  it  is  possible  that  the  con¬ 
ditional  probabilities  are  all  given  relative  to  some 
common  joint  probability  p^=p,  i-1,2,..,  in  which  case 

the  computational  task  simplifies. 

C7l  Simply  use  eq.(4.1).  An  example  of  a  very 

general  formula  encompassing  the  probability  evalua¬ 
tion  of  an  arbitrary  boolean  function  o.f  multiple 
conditional  events  (consider  again  the  comments  at 
the  beginning  of  Corollary  3.1)  is  given  in  (3.32). 

f9X  A  related  issue  to  the  actual  carrying  out  of 
the  above  steps  in  considering  data  fusion  problems 
is  that  nf  houndino  uncertainty  or  information.  One 
straightforv/ard  result  is  the  following,  utilizing 
the  results  of  section  3.3  and  the  well  known  FrSchet 
bounds  for  probabilities  (see,e.g.[l2]) : 

Th eorem  4.3. 

For  any  a^^b^  e  R,  j=l,..,n,  and  probability 
measure  p:R [0 ,1],  ^  (4,4) 

nisx(o,  '?p(a:|&bJ-{n-l))sp(  &(ajb  ))£  min  (p(a.|b.))  . 

niax(p(a.|b  ))sp(  v(a.|b  ))s  min(l,  J  p(a .  |  8b.)),(4.5) 

j=l . J  ^  ^  J=1  J  ^ 

Making  the  usual  definitions  for  information 
uncertainty  or  entropy ^  for  any  a,b  e  R,  p(b)>0, 

Hp(a|b)  =  -log(p(a|b))  ,  (4.6) 


it  follows  that  Theorem  4.3  can  be  converted  immedi¬ 
ately  to  bounds  on  information  uncertainty. 

Ideally,  what  is  sought  is  the  conditional 
event  calculus  -as  developed  here-  analogue  of  Hail- 
perin's  results  [12]  concerning  the  bounding  of  prob¬ 
ability  values  (and  hence,  correspondingly,  informa¬ 
tion  uncertainty)  for  arbitrary  boolean  functions  of 
unconditional  events  when  probability  bounds  are 
known  for  the  individual  (unconditional)  events.  How¬ 
ever,  at  this  point,  one  must  be  content  with  the 
rather  simple  results  given  in  Theorem  4.3,  until  fur¬ 
ther  results  are  obtained  in  this  area.  These  proper¬ 
ties  also  tie-in  with  CPL  and  the  weakened  forms  of 
implication  mentioned  in  section  3.5(i1)(XTI),  More 
details  of  this  will  be  presented  in  future  work. 

Generalizing  the  situation  described  in  steps 
^Pandf7A  one  can  have  events  of  interest  a.,b.  ,c.,d,, 
0.  (null  event),  ft.  (universal  event)  belong¬ 
ing  tJ  boolean  algebra’of  events  R.  corresponding  to 
common  probability  measure  p.  :R| -»■  TO  »1  ]»  Tor  1=1,2, 
say.  But,  by  making  the  usual  marginal  identifications 
a^=a.jXft2,  b^=b.|Xft2,.. ,82=11^x32, and  assum¬ 
ing  that  the  joint  probability  measure  p  of  Pj  and 
■^is  known,  noting  that  because  of  (4.1)? 


(ID.  Finally,  it  should  be  remarked  that  for  non- 
stochastic  information^,  such  as  that  containing  ling¬ 
uistic-based  evidence,  as  considered,  e.g.,  in  [11], 
an  analogous  calculus  of  conditional  forms  can  be  de¬ 
veloped,  based  upon  (3.11)  and  the  natural  extensions 
of  operations  [13].  The  corresponding  full  ALDP's 
should  prove  of  use  in  treating  combination  of  evi¬ 
dence  problems  and  data  fusion  in  general. 
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it  follows  that  all  of  the  previous  steps  are  now  val¬ 
id  for  the  situation  here  with  p^  replacing  p,  and  the 
marginal  identifications  for  the  events. 

flCOi  Higher  order  conditioning,  i.e.,  when  the 
events  in  the  above  steps  such  as  a,b,c,...  are  in 
actuality  in  conditioned  form  already  (which  may  well 
be  a  common  situation),  can  be  treated  in  a  straight¬ 
forward  way  by  use  of  the  results  in  section  3.5(iii), 
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ABSTRACT 

Electronic  support  measures  (ESM)  systems  are  passive  surveillance  sensors  for  determining  emitter  and  platform  information  from  a 
target’s  radio  frequency  emissions.  When  paired  with  a  radar  track  on  a  target,  this  information  can  be  valuable  in  identifying  it.  The  pairing 
could  be  done  manually  by  a  surveillance  operator  or  automatically  by  an  operational  computer  program, 

A  technique  for  automatically  correlating  ESM  reports  and  radar  tracks  has  been  developed  which  shows  the  potential  for  good  to 
excellent  performance  while  requiring  modest  data  processing  capabilities. 

The  approach  is  to  employ  a  recursive,  multiple  hypothesis  track-to-ESM  report  correlation  testing  technique  which  uses  probability 
thresholds  to  pmne  unlikely  hypotheses  over  a  number  of  association  attempts.  The  distinguishing  features  of  the  technique  are  its  adaptive 
azimuth  windows  which  are  used  in  data  association  and  the  decision  confidence  logics  which  are  used  to  winnow  unlikely  candidate 
hypotheses. 

Algorithms  for  this  technique  have  been  developed  and  their  performance  evaluated  by  simulation.  The  paper  outlines  the  salient 
features  of  the  correlation  process  and  presents  the  results  of  the  simulation. 


189 
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INTRODUCTION 


Increases  in  the  electronic  countermeasures  (ECM)  threat  to 
active  radar  and  identification  friend  or  foe  (IFF),  together  with 
advances  in  electronic  support  measures  (ESM)  technology  are 
making  passive  sensors  an  attractive  adjunct  to  active  sensors  for 
surveillance  and  identification  in  tactical  air  defense  systems.  Inte¬ 
grating  a  new  sensor,  such  as  ESM,  into  an  existing  tactical  com¬ 
mand  and  control  (C^)  system  can  present  a  number  of  special 
challenges  when  addressing  the  requirement  for  fusing  data  from 
the  newly  incorporated  sensor  and  an  existing  one. 

The  solution  to  the  fusion  problem  for  this  situation  must 
normally  satisfy  a  highly  constrained  set  of  conditions.  It  must 
contend  with  design  choices,  such  as  selection  of  tracking  coordi¬ 
nate  system,  filters  and  state  variables,  which  were  made  without 
consideration  for  a  multisensor  data  fusion  requirement.  Frequently, 
the  solution  must  be  capable  of  being  implemented  within  existing 
computer  and  operational  computer  program  architectures  and 
resources.  Above  all,  the  solution  must  provide  tactically  signifi¬ 
cant  information  in  time  for  the  user  to  take  advantage  of  it. 

This  paper  outlines  an  automated  fusion  technique  for  corre¬ 
lating  ESM  data  and  radar  tracks.  The  technique  has  the  potential 
for  achieving  good  to  excellent  performance  while  requiring  modest 
data  processing  capabilities.  In  principle,  the  technique  can  be 
implemented  in  a  processor  which  supports  an  automated,  real¬ 
time  radar  tracking  function. 

The  technique  is  applicable  to  both  airborne  and  ground  C~ 
systems  which  have  a  requirement  for  correlating  ESM  data  and 
radar  tracks.  The  more  general  airborne  application  is  presented 
here;  the  ground-based  application  can  be  considered  as  a  special 
case  in  which  there  is  no  ownship  motion. 

The  remainder  of  the  paper  discusses  the  problem  and 
describes  the  salient  features  of  the  fusion  technique.  The  results 
of  a  simulation  are  used  to  present  an  example  of  the  technique 
and  address  potential  performance. 


PROBLEM 


radar  tracker  considered  is  a  conventional  track-while-scan  (TWS) 
system  which  uses  Kalman  filtering  to  estimate  target  position  and 
velocity  in  a  Cartesian  coordinate  plane.  The  ESM  system  is  a 
general  purpose,  wide-band  passive  surveillance  sensor  for  deter¬ 
mining  angle  of  arrival  (AOA)  to  a  target  as  well  as  emitter  and 
platform  information  from  the  target’s  radio  frequency  (RF)  emis¬ 
sions.  If  an  ESM  report  could  be  paired  with  a  radar  track  on  a 
target,  the  emitter  and  platform  information  can  be  valuable  in 
identifying  it.  For  example,  the  ESM  report  could  indicate  the 
presence  of  a  hostile  fighter. 

The  hypothetical  air  picture  in  figure  1  indicates  the  nature 
of  the  correlation  problem.  The  airborne  system  is  at  the  center 
of  the  display.  Radar  returns  which  have  been  correlated  with 
tracks  surround  it  throughout  the  surveillance  volume  (all  tracking 
symbology  has  been  shut  off).  Reports  on  ESM  observations  are 


The  particular  problem  addressed  here  is  that  of  automati¬ 
cally  coirelating  radar  tracks  with  ESM  system  reports.  Both  sen¬ 
sors  are  taken  to  be  integrated  into  an  airborne  platform.  The 
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Figure  1 ,  Hypothetical  Radar  and  ESM  Air  Picture 


displayed  as  short  strobes  on  the  periphery  of  the  display.  Since 
angle  information  is  the  primary  discriminant  for  correlating  ESM 
reports  and  radar  tracks,  the  correlation  cannot  always  be  made 
unambiguously  or  with  high  confidence  in  one  attempt. 

Note  that  the  problem  addressed  here  is  correlating  the  cor¬ 
rect  radar  track  with  an  ESM  report  of  interest.  The  technique  dis¬ 
cussed  below  can  be  applied  to  the  converse  problem,  mutatis 
mutandis. 


CORRELATION  TECHNIQUE 

The  approach  is  to  use  a  recursive,  multiple  hypothesis  track- 
to-ESM  report  correlation  testing  technique  which  employs  proba¬ 
bility  thresholds  to  pmne  unlikely  hypotheses  over  a  number  of 
association  attempts. 

A  distinction  is  made  in  this  paper  between  association  and 
correlation.  Association  is  a  test  to  determine  whether  a  radar  track 
and  ESM  report  fall  in  an  azimuth  window.  Correlation  is  the  rec¬ 
ommended  assignment  of  a  radar  track  to  an  ESM  report.  Correla¬ 
tion  is,  therefore,  a  decision  which  is  made  on  the  basis  of  a 
number  of  association  attempts. 

The  azimuth  windows  used  in  each  association  attempt  are 
sized  adaptively  using  ESM  system  AOA  measurement  error  statis¬ 
tics  and  the  real-time  estimates  of  radar  tracker  performance  pro¬ 
vided  by  the  Kalman  filter  covariance  matrix.  This  careful  sizing 
of  the  association  windows  allows  the  accurate  prediction  of  the 
association  performance  between  the  correct  track  and  the  ESM 
report  over  a  number  of  association  attempts.  In  addition  to  win¬ 
nowing  unlikely  hypotheses,  the  association  performance  thresholds 
can  be  used  to  attribute  a  level  of  confidence  to  any  surviving 
hypotheses. 

The  major  functional  elements  of  the  correlation  technique 
are  initiation,  gating,  hypothesis  selection,  testing,  deletion  and 
hypothesis  declaration.  These  elements  are  described  below. 

Initiation 

Initiation  starts  the  correlation  process  by  specifying  ESM 
information  of  interest  and  a  nominal  AOA.  This  can  be  done  in 
many  ways.  One  way  would  be  for  the  surveillance  operator  to 
mark  a  particular  ESM  report  for  processing  by  console  switch 
actions.  Alternatively,  correlation  processing  could  be  automatically 
initiated  on  ESM  reports  passing  filters  for  geographical  sectors 
and  emitter  type. 

Gating 

The  purpose  of  gating  is  to  eliminate  the  vast  majority  of 
highly  unlikely  correlation  hypotheses.  After  the  correlation  process 
is  initiated,  the  radar  tracks  are  filtered  using  a  constant  width  azi¬ 
muth  window  centered  about  the  initiation  AOA.  Tracks  falling 
outside  the  window  represent  improbable  hypotheses  in  that  they 
are  unlikely  to  have  generated  the  ESM  report  and  are  dropped 
from  further  processing.  An  azimuth  window  width  of  from  5  to 
10  times  the  standard  deviation  of  the  combined  nominal  ESM 
AOA  measurement  error  and  angular  component  of  the  radar  track 
estimation  error  has  produced  consistently  good  gating  results. 

Hypothesis  Selection 

The  remaining  tracks  represent  tentative  correlation  hypothe¬ 
ses  which  are  subjected  to  a  more  rigorous  azimuth  association 
test.  Tracks  passing  this  test  are  promoted  to  candidate  hypothesis 
status.  This  test  uses  adaptive  azimuth  windows  which  account  for 
the  angular  component  of  the  radar  track  position  error  and  ESM 
report  AOA  error. 


For  each  track  hypothesis  making  it  to  this  point  in  the  pro¬ 
cessing,  the  positions  of  the  airborne  platform  and  the  radar 
track  are  linearly  predicted  and  the  1 -sigma  error  ellipse  defined 
by  the  position  elements  of  the  radar  track’s  covariance  matrix  are 
propagated  to  the  time  of  the  ESM  report.  The  predicted  positions 
are  used  to  calculate  a  predicted  track  azimuth  which  is  then  used 
to  determine  the  azimuthal  component  of  the  propagated  error 
ellipse.  This  part  of  the  processing  is  depicted  in  figure  2. 

In  the  particular  example  shown  in  figure  2,  the  current 
platform  position  information  (from  its  on-board  navigation  com¬ 
puter)  is  the  most  recent.  ESM  report  data  (from  the  ESM  subsys¬ 
tem  active  emitter  file)  are  next  and  the  smoothed  track  data  (from 
the  radar  track  table)  are  the  most  stale. 


As  depicted,  the  current  platform  position  coordinates 
(u,v)  are  predicted  back  in  time  to  (Up,  Vp)  using  the  navigation 
system  estimates  of  ownship  velocity.  The  position  coordinates  of 
the  track’s  smoothed  state  vector  (x^,  y^)  are  predicted  forward  in 
time  to  (Xp,  yp)  using  the  smoothed  velocity  components.  That  is: 

Up  -  u  -I-  uAt,  Vp  =  V  -t-  vAti 

Xp  =  Xs  +  XsAt2  yp  —  Ys  +  Ys  +  ys^l2 

where 

Atj  =  tgsM  -  t^AV  ”  IeSM  ~  Is- 

The  predicted  track  azimuth  is  calculated  as: 

Arctangent  (Xp  -  Up)/(yp  -  Vp). 

The  1 -sigma  position  error  ellipse  defined  by  the  track’s 
covariance  matrix  is  linearly  propagated  to  the  ESM  report  time 
according  to: 

where  P  and  P'  are  the  smoothed  and  predicted  covariances  and  ^ 
is  the  transition  matrix.  The  ellipses  are  also  depicted  in  figure  2. 

The  azimuthal  component  of  the  propagated  1 -sigma  error 
ellipse  is  then  calculated  from  the  predicted  azimuth  and  the  propa¬ 
gated  1 -sigma  error  ellipse  as: 

(y,  -  Vp)^  -  2(yp-  Vp)  (x,  -  Up)P'[l,21  +  (Xp  -  Up)P'[2,2] 

[(Xp  -  Up)^  +  (yp  -  Vp)^l^ 
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where 


P'[l,l]  =  P[l,l]  +  P[3,3](At)^  +  2P[l,3]At, 

P'[l,2]  =  P[l,2]  +  At(P[2,3]  +  P[l,4]  +  AtP[3,4]) 
and 

P'[2,2]  =  P[2,2]  +  P[4,4](At)^  +  2P[2,4]At. 

This  is  AZUN  which  represents  the  variance  of  the  predicted  track 
azimuth  error.  The  standard  deviation,  VaZUN,  is  depicted  in 
figure  2.  The  notation,  P[row,  column],  is  used  to  indicate  the 
elements  of  the  covariance  matrix  and  adheres  to  the  usual  conven¬ 
tions.  That  is,  P[i,i],  for  i  =  1,  .  .  .  ,  4,  is  the  variance  of  the 
state  variables  x,  y,  x,  y,  respectively. 

The  variance  of  the  track  azimuth  error  is  then  added  to  the 
variance  of  the  ESM  report  azimuth  error,  MEASUN,  to  calculate 
the  variance  of  the  combined  track  azimuth  and  ESM  report  azi¬ 
muth  errors.  This  combined  variance  is  used  to  size  an  azimuth 
association  window,  CORWIN,  according  to: 

CORWIN  =  2  X  V(AZUN  +  MEASUN). 

The  association  window  is  then  centered  about  the  ESM 
report  AOA  as  depicted  in  figure  3.  If  the  predicted  track  position 
falls  in  the  association  window,  the  track  is  tentatively  paired  with 
the  ESM  report  and  an  association  counter  is  initialized  for  that 
track.  Note  that  under  the  assumption  that  the  azimuth  errors  in 
the  radar  track  and  ESM  report  are  independent  and  normal  with 
zero  mean,  CORWIN  represents  a  2-standard  deviation  window 
for  a  zero  mean,  normal  probability  distribution.  As  a  result,  the 
correct  track  can  be  expected  to  fall  in  the  association  window 
about  95  percent  of  the  time. 
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MEASUN  is  the  variance  of  the  ESM  report  AOA  error. 
ESM  AOA  measurement  performance  is  normally  amenable  to 
table  look-up  techniques  for  the  purpose  of  building  correlation 
windows.  Specific  values  for  the  table  would  be  developed  from 
ESM  system  specification  requirements  or  test  data. 

Tracks  which  associate  with  the  ESM  report  at  least  once  in 
the  first  three  attempts  are  promoted  to  candidate  hypothesis  status. 
Using  this  criterion,  the  only  way  the  correct  track  would  not  be 
promoted  is  if  it  failed  to  pass  all  three  association  attempts.  Under 
the  assumption  that  the  association  attempts  are  independent  events, 


then  the  probability  of  missing  all  three  is  (1.0  -  .95)^  =  1.25  X 
10"^.  The  probability  of  the  complementary  event,  promoting  the 
correct  track,  is  therefore  1.0  -  1.25  X  lO"^  «  0.99. 

Hypothesis  Testing  and  I>eletion 

After  they  are  promoted,  candidate  track  hypotheses  are  pro¬ 
cessed  against  the  ESM  report  over  time  to  generate  association/ 
missed-association  histories.  This  is  done  by  using  the  technique 
just  described  in  the  hypothesis  selection  section. 

The  association  history  of  a  hypothesis  begins  with  process 
initiation.  The  history  ends  after  a  predefined  maximum  time  or 
when  a  hypothesis  is  eliminated  as  being  unlikely.  After  each  asso¬ 
ciation  cycle,  the  association  histories  of  the  hypotheses  are 
updated  to  reflect  the  most  recent  outcome.  The  probabilities  of 
the  updated  hypotheses  are  recomputed  and  used  to  evaluate 
whether  a  hypothesis  is  eliminated  or  retained.  This  is  the  recursive 
feature  of  the  correlation  technique.  Two  criteria  are  used  to  elimi¬ 
nate  candidate  hypotheses:  exceeding  a  cumulative  missed  associa¬ 
tion  threshold  or  three  consecutive  missed  associations. 

The  cumulative  miss  threshold  represents  a  level  of  decision 
confidence  for  retaining  or  eliminating  candidate  hypotheses  over  a 
sequence  of  association  attempts.  The  threshold  is  determined  by 
estimating  the  number  of  times  the  correct  track  can  be  expected 
to  fall  in  the  association  window  of  an  ESM  report  over  a  sequence 
of  association  attempts.  Under  the  assumption  that  individual  asso¬ 
ciation  attempts  are  independent  events,  the  expected  number  of 
associations  over  multiple  attempts  is  a  binomial  random  variable. 
The  expected  number  of  associations  can  be  calculated  from  the 
expected  performance  of  a  single  attempt  and  the  probability  func¬ 
tion  of  the  binomial  distribution  according  to: 

b(x;n,  p)  =  (  ^  )p\l  -p)"-* 

where  X  is  the  number  of  successful  associations,  n  is  the  number 
of  association  attempts  and  p  is  the  probability  of  correct  associa¬ 
tion  for  a  single  attempt.  Strictly  speaking,  the  assumption  of  inde¬ 
pendence  between  successive  association  attempts  is  not  valid 
because  of  the  error  correlation  between  radar  track  estimates  from 
smoothing  to  smoothing.  In  practice,  however,  this  assumption  has 
negligible  impact  on  correlation  performance  because  of  the  relative 
magnitude  of  the  ESM  report  and  radar  track  errors. 

To  see  how  the  cumulative  miss  threshold  is  computed  and 
used,  consider  the  previously  defined  2-sigma  association  windows. 
Recall  that,  with  this  window,  there  is  a  95%  probability  of  associ¬ 
ating  the  correct  track  with  the  ESM  report  in  a  single  attempt. 
Using  the  binomial  probability  function  and  noting  that  the  event 
“at  least  13  associations  in  15  attempts”  is  equivalent  to  the  logical 
union  of  the  three  mutually  exclusive  events  “exactly  x  associations 
in  15  attempts”  forx  =  13,  14,  and  15,  there  is  97%  probability 
that  the  correct  radar  track  will  associate  with  the  ESM  report  at 
least  13  times  in  15  attempts  (two  or  fewer  misses).  A  track 
hypothesis  processed  at  the  97%  confidence  level,  therefore,  would 
be  eliminated  if  the  third  missed  association  occurred  at  the  fif¬ 
teenth  association  attempt  and  if  it  had  not  been  eliminated  earlier. 
Hypotheses  processed  at  higher  levels  of  confidence  (e.g.,  99%) 
would  require  more  misses  before  being  eliminated;  hypotheses 
processed  at  lower  levels  (e.g.,  90%)  would  be  eliminated  with 
fewer  misses. 

An  attractive  feature  of  this  approach  is  that  the  missed  asso¬ 
ciation  thresholds  corresponding  to  a  confidence  level  can  be  pre¬ 
computed  and  employed  in  table  look-up  techniques  rather  than 
computed  in  real-time.  An  example  for  the  95%  confidence  level 
is  shown  in  Table  1 .  A  hypothesis  would  be  evaluated  after  each 
association  attempt  and  eliminated  if  it  exceeded  the  permissible 
number  of  missed  associations. 
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Table  1.  Missed  Association  Thresholds  for  a  95%  Level 
of  Confidence 


Association  Attempt 

Permissible  Missed 
Associations 

1 

0 

2 

1 

3 

1 

4 

1 

5 

1 

6 

1 

7 

1 

8 

2 

9 

2 

10 

2 

11 

2 

12 

2 

13 

2 

14 

2 

15 

2 

16 

2 

17 

3 

18 

3 

19 

3 

20 

3 

The  three  consecutive  miss  criterion  is  an  ad  hoc  technique 
wJiich  has  been  employed  to  handle  situations  in  which  the 
i^ssed  association  history  of  an  incorrect  hypothesis  fell  far 
below  threshold,  making  it  unlikely  to  be  eliminated  quickly. 

This  can  occur  when  the  airborne  platform,  the  correct  track 
and  an  incorrect  track  are  colinear  for  some  time.  The  three 
consecutive  miss  criterion  could  eliminate  an  incorrect  hypothesis 
quickly  while  posing  a  small  risk  of  eliminating  the  correct 
hypothesis. 

Declaration 

Processing  candidate  hypotheses  continues  until  one  of  the 
following  conditions  is  satisfied: 

1.  After  a  predefined  minimum  number  of  processing  cycles, 
all  but  one  candidate  hypothesis  is  eliminated,  or 

2.  Processing  has  exceeded  a  predefined  maximum  time  or 
number  of  processing  cycles,  or 

3.  All  candidate  hypotheses  are  eliminated. 

If  the  first  condition  is  satisfied,  the  hypothesis  is  accepted  and  the 
surveillance  operator  is  notified  of  the  recommended  track-to-ESM 
report  correlation.  If  the  second  condition  holds,  there  are  multiple 
hypotheses  remaining  and  the  operator  is  notified  of  the  ambiguous 
correlations.  If  the  third  condition  holds,  the  operator  is  notified 
that  no  correlations  were  made. 


AN  EXAMPLE 

Returning  to  the  hypothetical  air  picture  depicted  in  figure 
1 ,  assume  the  ESM  report  marked  by  the  arrow  unambiguously 
indicates  a  hostile  fighter.  There  are  a  number  of  radar  tracks 
which  could  reasonably  be  paired  with  the  ESM  report. 

.  After  correlation  processing  was  initiated  on  this  ESM  report, 
all  but  eight  tracks  were  eliminated  by  gating.  These  eight  tracks 
are  depicted  notionally  in  figure  4  along  with  their  velocity  vectors. 
Association  attempts  were  made  every  five  seconds,  corresponding 


Figure  4.  Hypothetical  Correlation  Problem 

to  the  refresh  rate  which  was  posited  for  the  ESM  system.  Tracks 
1,  2  (the  correct  track)  and  7  each  associated  at  least  one  time  in 
the  first  three  attempts  and  so  became  candidate  hypotheses.  Test¬ 
ing  these  hypotheses  at  the  95  percent  confidence  level,  track  1 
was  eliminated  35  seconds  after  process  initiation  for  exceeding 
the  cumulative  miss  threshold  and  track  7  was  eliminated  10  sec¬ 
onds  later  for  exceeding  the  cumulative  miss  threshold  and  three 
consecutive  misses  simultaneously.  Track  2  was  therefore  uniquely 
and  correctly  paired  with  the  ESM  report  in  45  seconds. 

PERFORMANCE  TRENDS 

The  automated  process  was  simulated  for  each  of  the  eight 
ESM  reports  in  figure  4  over  a  range  of  ESM  AOA  accuracies  and 
confidence  levels  for  eliminating  candidate  hypotheses. 

Certain  ground  mle  assumptions  applied  to  the  simulation. 
Targets  did  not  maneuver  during  the  scenario.  The  radar  antenna 
was  assumed  to  rotate  in  azimuth  mechanically  with  a  10  second 
period.  Radar  detection  performance  was  assumed  to  be  adequate 
to  support  tracking  using  conventional  Kalman  filtering  techniques. 
The  tracker  predict-correlate-smooth  cycle  was  assumed  to  be  10 
seconds  corresponding  to  the  radar  antenna  rotation  period. 

The  tracking  filter  was  taken  to  be  implemented  as  two  inde¬ 
pendent  channels  —  one  for  the  x-variables  and  one  for  the  y- 
variables.  This  is  the  approach  frequently  taken  in  real-time 
applications  where  any  loss  in  track  estimation  accuracy  is  compen¬ 
sated  by  a  significant  reduction  in  processing  required  to  execute 
the  filtering.  Note  that,  in  this  approach,  the  cross-correlation  ele¬ 
ments  of  the  fully  coupled  covariance  matrix  are  not  computed  and 
so  are  unavailable  for  determining  AZUN.  No  compensation  was 
made  for  this  in  sizing  the  association  windows.  As  a  result,  the 
cross-correlation  elements  were  set  to  zero  when  computing  AZUN 
in  the  simulation. 

Regarding  the  ESM  system,  each  target  was  assumed  to 
emit  a  signal  throughout  the  scenario  that  was  detected  by  the  ESM 
system  and  was  distinguishable  from  the  other  target  signals.  It 
was  assumed  that  the  ESM  system  could  be  directed  to  search  the 
RE  spectrum  of  interest  so  that  emitter  reports  in  the  active  emitter 
file  would  be  updated  every  5  seconds.  Actual  update  times  were 
modeled  as  occurring  randomly  in  each  5  second  update  cycle  to 
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account  for  changes  in  the  ambient  emitter  activity,  the  sequence 
and  timing  of  the  ESM  system  receiver  search  strategy  and  signal 
processing  activities. 

The  radar  and  ESM  system  were  assumed  to  operate  inde¬ 
pendently.  As  a  result,  there  was  no  time  synchronization  between 
the  update  of  the  radar  track  file  and  the  ESM  system  active  emit¬ 
ter  file. 


Correlation  performance  is  summarized  in  figure  5.  There  is 
a  performance  curve  for  each  level  of  ESM  AOA  accuracy  consid¬ 
ered.  In  the  majority  of  scenarios,  seven  of  eight  radar  tracks  were 
uniquely  correlated  with  the  correct  ESM  report  and  so  average 
time  to  correct  correlation  is  used  to  indicate  the  relative  perfor¬ 
mance  among  the  AOA  accuracies  and  confidence  levels  consid¬ 
ered.  The  data  for  scenarios  in  which  seven  of  eight  unique,  correct 
coirelations  were  achieved  are  joined  by  solid  lines.  The  one  sce¬ 
nario  in  which  the  correct  track  was  eliminated  and  the  two  sce¬ 
narios  in  which  six  of  eight  unique,  correct  correlations  were 
achieved  are  included  for  completeness.  These  results  are  joined  to 
the  other  data  by  broken  lines.  Correlation  performance  was 
degraded  in  these  scenarios  and  so  average  time  to  correct  correla¬ 
tion  is  not  an  adequate  indicator  of  relative  performance. 


L  NOTE  1  ^  — 


'  90  9S  98  99 

LEVEL  OF  DECISION  CONFIDENCE  (PERCENT) 

NOTE  1  -  ONE  CORRECT  TRACK  ELIMINATED  IN  THIS  SCENARIO 

NOTE  2  -  SIX  OF  EIGHT  CORRECT,  UNIQUE  PAIRINGS;  2  CORRECT,  AMBIGUOUS  PAIRINGS 

Figure  5.  Correlation  Performance  Summary 


Requiring  veiy  high  confidence  in  the  results  increased  the  number 
of  ambiguous  correlations  in  this  case. 

At  the  95%  level  of  confidence,  the  final  correlation  results 
for  the  2.0  degree  accuracy  scenarios  were  the  same  as  those  for 
the  0.5  and  1 .0  degree  scenarios  —  seven  of  eight  unique,  correct 
correlations.  The  average  time  to  achieve  the  results  was  159  sec¬ 
onds  compared  to  125  seconds  for  the  1.0  degree  scenario  and  52 
seconds  for  the  0.5  degree  scenario. 

The  correlation  results  did  not  differ  at  the  98%  level  of 
confidence  although  the  average  time  to  achieve  them  increased  to 
more  than  four  minutes  (247  seconds).  At  the  99%  level,  the 
hypothesis  which  paired  track  2  with  ESM  report  4  could  not  be 
eliminated.  This  decreased  the  unique  correlations  from  seven  to 
six  and  increased  the  ambiguous  ones  from  one  to  two. 

The  2.0  degree  scenario  demonstrated  that  processing  at 
lower  levels  of  confidence  increases  the  risk  of  eliminating  the 
correct  track.  At  the  90%  level,  the  correct  hypothesis  which 
paired  track  4  with  ESM  report  4  was  eliminated  for  exceeding  the 
cumulative  miss  threshold.  At  that  point,  the  only  hypothesis 
regarding  ESM  report  4  which  remained  was  the  one  pairing  track 
2  with  it,  although  this  hypothesis  would  have  been  eliminated  15 
seconds  later,  also  for  exceeding  threshold. 

In  general,  the  better  the  ESM  AOA  accuracy,  the  better  the 
correlation  performance,  but  acceptable  performance  seems  likely 
up  to  2.0  degrees  AOA  accuracy.  Note  that  these  results  were 
achieved  without  compensating  for  the  unavailable  cross-correlation 
elements  of  a  fully  coupled  Kalman  filter  or  the  invalid  assumption 
of  independence  between  association  attempts.  In  general,  these 
would  have  to  be  considered.  In  this  application,  however,  they 
were  of  negligible  impact  because  the  ESM  AOA  errors  generally 
dominated  the  radar  track  AOA  errors. 

Selecting  a  confidence  level  for  processing  involves  striking 
a  balance  between  tolerating  some  ambiguous  or  incorrect  correla¬ 
tions  and  timeliness  of  results.  This  balance  may  ultimately  be 
decided  by  operational  requirements  in  specific  situations.  How¬ 
ever,  the  95  %  level  of  confidence  seemed  to  be  the  best  compro¬ 
mise  in  the  examples  considered  in  this  paper. 


SUMMARY 


Several  trends  are  indicated.  Good  to  excellent  correlation 
results  appear  achievable  over  the  ranges  of  confidences  and  AOA 
accuracies  considered. 

The  correlation  results  were  the  same  for  the  four  cases  in 
which  ESM  AOA  accuracy  was  0.5  degrees  (1 -sigma).  There  were 
seven  of  eight  unique,  correct  correlations.  However,  as  the  confi¬ 
dence  required  to  make  these  correlations  increased,  so  did  the 
time  to  achieve  them.  The  average  time  ranged  from  48  seconds  at 
the  90%  confidence  level  to  85  seconds  for  the  99%  confidence 
level. 

The  final  correlation  results  for  the  1.0  degree  AOA  accuracy 
scenarios  were  the  same  as  those  for  the  0.5  degree  scenarios  at 
the  90,  95,  98  percent  levels  of  confidence.  There  were  seven  of 
eight  unique,  correct  correlations.  However,  the  average  time  to 
achieve  these  results  increased.  For  example,  at  the  95%  confi¬ 
dence  level  it  took  125  seconds  compared  to  52  seconds  for  the 
0.5  degree  scenario.  More  time  was  required  for  the  targets  to 
separate  enough  to  allow  the  incorrect  tracks  to  be  confidently 
disassociated  from  the  1 .0  degree  accuracy  ESM  reports.  At  the 
99%  confidence  level,  the  hypothesis  which  paired  track  2  with 
ESM  report  4  could  not  be  eliminated.  Therefore,  six  of  eight 
tracks  were  uniquely  and  correctly  correlated  with  their  ESM 
reports  while  two  were  correctly  but  ambiguously  correlated. 


This  paper  has  outlined  an  automated  technique  for  correlat¬ 
ing  ESM  data  and  radar  tracks  which  has  the  potential  for  achiev¬ 
ing  good  to  excellent  performance  while  requiring  modest  data 
processing  capabilities.  It  is  a  suitable  baseline  for  C‘  systems 
where  the  fusion  capability  must  be  implemented  with  nominal 
computer  resources. 

The  technique  is  a  recursive,  multiple  hypothesis  track-to- 
ESM  report  correlation  algorithm  which  employs  probability 
thresholds  to  pmne  unlikely  hypotheses  over  a  number  of  associa¬ 
tion  attempts.  The  association  windows  are  carefully  sized  to  elimi¬ 
nate  unlikely  hypotheses  and  attribute  a  level  of  confidence  to 
surviving  hypotheses. 

A  distinctive  feature  of  the  technique  is  the  capability  to  set 
the  level  of  confidence  at  which  processing  is  performed.  Setting 
of  the  confidence  level  could  be  implemented  as  an  adaptation 
parameter  which  would  be  set  to  a  default  value  or  selected  by  the 
operator  before  or  during  a  mission.  This  permits  the  human  to 
make  the  important  operational  decision  which  strikes  the  balance 
between  timeliness  of  results  and  tolerating  some  ambiguous  or 
incorrect  correlations  while  leaving  the  execution  of  the  correlation 
function  to  an  operational  computer  program. 
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ABSTRACT 

Effective  weapon  and  surveillance  systems  require  accurate  spatial  location  Information  for 
tactical  functions  such  as  fire  control  and  early  warning.  Jamming,  adverse  weather,  and  de¬ 
ceptive  target  manuevers  often  found  in  the  air/land  combat  environment  present  a  challenge  In 
determining  a  target*s  position  in  space.  Multi— sensor  fusion  can  Improve  the  reliability  of 
the  tracking  system;  because  of  their  mu  1 1 i- spect ral  nature,  such  systems  are  more  immune  to 
unfavorable  environmental  conditions.  The  fusion  of  2-D  passive  sensors  with  3-D  active  sensors 
can  provide:  1)  more  covert  surveillance,  2)  better  target  identification,  3)  Improved  target 
track-file  association,  4)  higher  target  detection  probability  with  fewer  false  alarms,  and  5) 
more  precise  estimates  of  the  3-D  target  position.  This  study  focused  primarily  on  the  last 
advantage  of  sensor  fusion,  i.e.,  5).  Sensor  Integration  is  considered  in  terms  of  fusing  two- 
and  three-dimensional  spatial  measurements.  The  resulting  Improvement  in  estimating  3-D  position 
provides  a  quantitative  metric  of  fusion  performance.  Significant  performance  improvement  in 
tracking  accuracy  achieved  in  a  simulation  for  both  helicopter  and  fixed-wing  targets  is  reported 
in  this  paper. 
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INTRODUCTION 

Emerging  military  systems  require  opti¬ 
mal  use  of  multiple  sensors  and  information 
sources  In  order  to  meet  mission  objectives. 
Their  survivability  must  be  enhanced  through 
longer  standoff,  significantly  reduced  false 
alarms,  improved  target  detection  and  iden¬ 
tification,  and  operator  workload  reduction. 
These  are  all  key  to  reducing  mission  time¬ 
line  and  consequent  platform  exposure  (Ref.l). 
Algorithms  and  processing  techniques  that  ac¬ 
complish  the  synergistic  combining  of  sensor 
inputs  across  the  frequency  spectrum  (acous¬ 
tic,  IR,  TV,  radar),  as  well  as  stored  data 
inputs  such  as  digital  map,  have  not  yet 
reached  operational  maturity. 

This  paper  presents  an  algorithm  develop¬ 
ment  and  evaluation  effort  for  a  multi-sensor 
fusion  simulation  system.  This  VAX-based  sys¬ 
tem  is  easily  adapted  to  handle  a  wide  spec¬ 
trum  of  tactical  applications  such  as  those 
found  in  ground-to-air  and  air-to-air  scenar¬ 
ios.  Simulation  results  presented  here  focus 
on  a  multi-sensor  three-dimensional  tracking 
solution  in  an  air  defense  scenario. 


SENSOR  FUSION  SIMULATION  SYSTEM  AND  ALGORITHM 
DESCRIPTION  — 

The  sensor  fusion  simulation  system  shown 
in  Figure  1  has  five  major  functions:  report 
generation;  association;  false— alarm  reduc¬ 
tion;  ID  improvement;  and  tracking.  We  inves¬ 
tigated  candidate  approaches  for  the  last  four 
functions  and  selected  algorithms  for  imple¬ 
mentation  based  on  performance  and  processing 
complexity. 

Sensor  Report  Generation 

We  selected  a  sensor  suite  that  would 
be  applicable  to  an  integrated  air  defense 
system.  This  sensor  suite  was  composed  of 
radar,  infrared  search  and  track  (IRST),  ra¬ 
dio  frequency  direction  finder  (RFDF),  and  a 
nonimaging  (NI)  sensor.  Mission  profiles  for 
three  target  types  in  a  simulated  scenario 
were  established  using  the  Texas  Instruments 
developed  Composite  Air-Surface  Engagement 
Simulation  (CASES),  a  computer  engagement 
model  that  was  designed  to  evaluate  the  ef¬ 
fectiveness  of  su rf a ce— bas ed  air  defenses, 


air  vehicle  survivability,  and  al r-to-surf ace 
weapon  and  sensor  requirements.  Sensor  re¬ 
ports  for  each  of  six  sensor  sites  were  then 
generated  based  on  target  and  sensor  perform¬ 
ance  parameters. 
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Figure  1,  Sensor  Fusion  Simulation  System 


Association 

The  association  function  (Ref.  9)  is  a 
front-end  process  that  matches  each  of  the 
incoming  sensor  reports  with  the  fused  tracks 
already  in  the  track  file  database.  Its  de¬ 
cision  outputs  declare  one-to-one,  multiple, 
or  no  match.  Ambiguities  from  multiple  or 
no  matches  will  be  resolved  in  the  false- 
alarm  reduction  function. 

The  procedure  calls  for  a  common  refer¬ 
ence  transformation  (CRT)  in  both  space  and 
time  and  a  hypothesis  generation/evaluation/ 
selection  process.  The  CRT  allows  Intersys¬ 
tem  cueing,  noncollocated  sensor  deployment, 
and  Integration  flexibility  of  nonde velopmen— 
tal  Item  (NDI)  elements.  A  hypothesis  set  is 
generated  by  a  spatial  distance  thresholding 
criterion.  ID  and  kinematics  from  sensors, 
along  with  the  a  priori  sensor  performance, 
define  a  rule-based  expert  that  evaluates 
and  selects  association  hypotheses. 

False-Alarm  Reduction 

Four  false-alarra  reduction  algorithms  were 
considered  for  the  fusion  system:  Shafer-Demp- 
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Bter  (Ref.  6),  joint  statistical  detection 
(e.g.,  Bayesian),  Pd/Pf  ratio  (Pd  =  probabil¬ 
ity  of  a  detection,  Pf  =  probability  of  a 
false  alarm),  and  M-or-more  detection.  Based 
on  performance  and  processing  capabilities, 
the  last  approach  was  chosen  since  It  pro¬ 
vides  a  simple  solution  when  noise  Is  uncor¬ 
related  across  sensors.  This  was  a  reason¬ 
able  assumption  for  the  sensor-suite  selected 
in  this  study. 

Multi-sensor  detection  performance  can  be 
modeled  as  N  repetitions  of  Bernoulli  trials 
(where  N  is  the  number  of  sensors).  If  all 
N  trials  (sensors)  are  independent  and  the 
probabilities  (detection  and  false  alarm)  are 
the  same  (i.e.,  N  Identical  but  distinct  sen¬ 
sors),  the  detection  performance  of  the  N 
sensors  can  be  described  by  the  binomial  dis¬ 
tribution.  If  the  probabilities  are  not  the 
same  but  are  constant  (l.e.,  N  different 
sensors),  the  detection  performance  of  the  N 
sensors  can  be  described  by  a  generalization 
of  the  binomial  distribution.  In  either  case, 
the  question  that  seems  most  appropriate  is 
"Did  M  or  more  sensors  detect?".  If  they  did, 
a  detection  is  reported;  otherwise,  there  is 
no  detection. 

Our  results  showed  that  the  best  choice 
for  M  is  approximately  N/2;  i.e.,  if  half  or 
more  sensors  detect,  then  the  fused  decision 
is  a  detection.  This  fused  decision  will  al¬ 
ways  have  improved  detection  and  false— alarm 
performance  (l.e.,  increased  Pd/Pf  ratio)  com¬ 
pared  wltTi  any  of  the  individual  sensors. 

Identification  Improvement 

ID  improvement  occurs  at  three  levels. 
The  first  level  is  an  automatic  fusion  pro¬ 
cess  of  ID  data  from  the  NI  sensor  reporting 
rotary-wing  targets  and  the  RFDF  identifying 
radiating  targets;  both  sensors  are  capable 
of  identifying  multiple  targets.  At  the  next 
level,  some  high-priority  targets  that  need 
further  ID  clarification  are  processed  by 
other  dedicated  ID  sensors,  such  as  FLIR  and 
other  nonimaging  ID  sensors,  that  may  be  in¬ 
cluded  in  the  sensor  suite;  the  dedicated 
sensors  must  be  cued  for  each  target.  The 
third  decision  is  made  at  the  operator  level; 
the  operator  evaluates  the  machine  recommen¬ 
dations,  with  aids  from  other  intelligence 
data  about  the  target,  to  derive  the  final 
decis i on , 


available.  Passive  sensors  offer  no  range 
information,  and  some  tactical  situations 
may  not  warrant  operation  of  radar,  even  i 
it  is  available.  Jamming  and  target  inter- 
vlsiblllty  probl'ems  may  also  restrict  range 
measurements.  The  tracking  approach  must  be 
flexible  enough  to  incorporate  passive  bear¬ 
ing-only  measurements  with  range  measurements 
whL  they  are  available.  For  these  reasons 
a  simple  alpha-beta  filtering  appproach  that 
provides  flexibility  in  system  design  and 
facilitates  a  direct  evaluation  of  the  ef¬ 
fects  of  fusing  passive  bearing-only  sen- 
.. _ _ 4o  here. 


Integrated  Tracker  Approach 


The  multi-sensor  tracking  system  has  sev¬ 
eral  features:  1)  angle  reports  from 
sensors  that  occur  simultaneously  in  time  are 
converted  into  a  single  fused  angle  report, 
2)  azimuth  and  elevation  reports  are  tracked 
independently  when  no  range  information  is 
available;  3)  when  radar  measurements  are 
available,  the  polar  information  is  trans¬ 
formed  into  the  Universal  Transverse  Mercator 
(UTM)  cartesian  reference  frame  and  tracked 
with  an  uncoupled  3-D  alpha-beta  tracker  ;  and 
4)  cartesian  position  is  transformed  back  into 
the  polar  reference  frame  and  the  smoothed 
bearing  estimates  are  used  to  update  the  angle 
tracker  described  in  2). 

Figure  2  is  a  block  diagram  of  the  multi¬ 
sensor  tracking  system.  First,  simultaneous 
bearing  reports  from  sensors  are  fused.  Math¬ 
ematically,  this  corresponds  to  forming  a 
minimum  mean  square,  unbiased  estimator,  X, 
of  target  bearing  given  simultaneous  sensor 
measurements,  Zi,  of  different  variances  (sen- 
si  tivitles  )  Assuming  random,  independent, 

unbiased  measurement  errors,  the  solution  is: 


x=  E 

1-1 


Ki  Zi 


(1) 


where 


Xl 


”  1 


j“l 


(2) 


The  error  of  the  estimate  is: 


Several  ID  decision  techniques  were  in¬ 
vestigated  as  an  aid  to  the  operator  for  the 
first-level  and  third-level  processes.  The 
techniques  include  Shaf er-Demps ter  rules  of 
combination  (Ref.  6),  joint  statistical  clas¬ 
sification,  and  rule-based  expert.  The  latter 
was  chosen  because  of  its  capabllltlty  to 
handle  incomplete  sensor  reports  while  making 
a  list  of  prioritized  probable  IDs  to  the 
operator. 


INTEGRATED  TRACKING  SOLUTION 
Track  Filter  Selection 

Several  multiple-target ,  multiple-sensor 
tracking  approaches  have  been  reported  in  the 
literature  (Refs.  2-8).  Nearly  optimal  track¬ 
ing  performance  can  be  achieved  with  the  Kal¬ 
man  class  of  filters,  depending  on  the  target 
.maneuver  model  and  the  specific  target  man¬ 
euver;  however,  to  implement  a  3-D  optimal 
Kalman  filter,  all  six  target  states  must  be 
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n 


(3) 


IIME  SEQUENTIAL 
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Figure  2.  Block  Diagram  of  Multi-Sensor 
Tracking  System 
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The  next  block  represents  a  bearing-only 
tracker.  Here,  the  time-sequential  bearing 
estimates  are  fused  using  a  1-D  alpha-beta 
track  filter.  The  weights  (alphas  and  betas) 
are  chosen  l^as ed  on  the  measurement  errors 
(l.e.,  the  )  .  The  output  of  this  tracker 
is  a  smoothed  estimate  of  the  target's  bearing 
(azimuth  and  elevation). 

When  radar  measurements  are  available, 
3-D  target  position  is  updated.  Measured 
range  and  range-rate  are  combined  with  the 
improved  bearing  estimates  from  the  bearing- 
only  tracker  with  a  pol ar- t o-car tes i an  trans¬ 
form.  The  target's  position  in  UTM  coordi¬ 
nates  is  estimated  and  tracked.  The  cartesian 
state  vector  is  then  transformed  back  to  a 
polar  vector,  and  the  3— D  bearing  values  are 
input  into  the  bearing-only  tracker  to  reduce 
angle  drift. 


MULTI-SENSOR  TRACKING  SIMULAT ION 


Sensors  and  Targets  Assumptions 


The  simulation  sensor  suite  consists  of  a 
radar  along  with  three  passive  sensors:  IRST, 
NI ,  and  RFDF .  To  simplify  the  simulation,  the 
timelines  of  all  of  the  sensors  are  assumed 
to  be  synchronized,  although  each  with  its  own 
update  rate  (4.0,  1.5,  1.0,  and  1.0  seconds, 
respectively).  We  assume  that  the  sensors 
are  collocated,  the  targets  are  not  severely 
manueverlng,  the  jamming  environment  is  be¬ 
nign,  and  the  Hind— D  gunship  has  a  multi- mode 
fire  control  radar  in  its  undernose  sensor 
pack . 


Since  the  simulation  baseline  and  3-D 
tracking  performance  is  primarily  driven  by 
the  radar  t rack-whi le— s can  requirements,  some 
of  the  radar  assumptions  are  discussed  here. 
The  antenna  is  assumed  to  be  a  flat-plate  1 
meter  in  elevation  by  2  meters  in  azimuth. 
The  operating  frequency  is  2  ghz.  Monopulse 
tracking  in  azimuth  and  elevation  is  assumed, 
yielding  effective  azimuth  and  elevation  beam- 
widths  of  0.5  degree  and  1  degree,  respec¬ 
tively.  The  radar  is  assumed  to  be  forward- 
looking  (180  degrees  in  azimuth)  and  re¬ 
stricted  between  +45  and  -45  degrees  in  ele¬ 
vation.  This  gives  450  spatially  distinct 
beams  (i.e.,  potential  targets)  that  can  be 
tracked.  The  PRF  is  7  .5  khz  based  on  the 
maximum  range  of  the  radar  (20  km).  The 
range  resolution  is  30  meters  and  the  range 
rate  resolution  is  2,5  meters  per  second. 


Sensor  and  Target  Models  for  Reports  Database 
Generation 


Table  1  contains  the  performance  char¬ 
acteristics  of  the  sensor  suite  in  terms  of 
sensor  update  rate,  detection  range,  pointing 
accuracy,  range,  and  range  rate.  As  depicted 
in  Figure  3,  sensor  reports  were  generated  in 
a  simulated  scenario  in  which  a  Flogger,  a 
Frogfoot,  and  a  Hlnd-D  flew  the  projected  pro¬ 
files  in  the  Defense  Mapping  Agency's  (DMA's) 
Fulda  Gap  map  area.  The  Flogger  flew  200 
meters  above  ground  level  ( AGL )  at  450  knots, 
while  the  helo  flew  50  meters  AGL  at  80  knots 
and  occasionally  at  treetop  levels.  These 
profiles  were  a  compromise  between  the  opera¬ 
tional  requirements  and  conditions  under  which 
a  meaningful  data  set  of  sensor  reports  can 
be  derived  (the  helo  flying  below  30  meter 
AGL,  for  example,  did  not  have  line  of  sight 
with  the  sensors  for  the  entire  route).  Both 
the  elevation  and  cultural  features  were  used 


to  determine  the  target  and  sensor  Intervls- 
Iblllty.  Sensor  inaccuracies  were  modeled 
with  lowpass-f iltered  white  noise.  In  this 
way,  a  data  set  of  sensor  reports  for  the 
Hlnd-D  and  the  Flogger  for  time  periods  of 
1133  and  101  seconds,  respectively,  was  gen¬ 
erated  for  the  multi-sensor  tracking  simula¬ 
tion  evaluation. 


Table  1 .  Sensor  Performance  Model 
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MULTI-PLATFORM  DATA  ARE  ORGANIZED  IN 


THESE  SYNTHETIC  DATA  BASES  ALLOW  SIMULATIONS  OF  INTER-PLATFORM 
CONFIGURATIONS  FOR  APPLICATIONS  AS  FAADC^I  - 


•  ASSUMPTIONS 
—BENIGN  JAMMING 
—RADAR-EQUIPPED  HtND-D  GUNSHIP 

—COLLOCATED  SENSORS  (WILL  EXTEND  TO  NONCOLLOCATED  CASES) 


Figure  3.  Sensor  Report  Generation 
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SIMULATION  RESULTS 

Three  Dimensional  Tracking  Results 

Simulation  results  for  tracking  a  fixed 
wing  target  are  shown  In  Figure  5(A):  the 

topmost  curve  represents  the  timeline  for  the 
radar  (high  signifies  that  the  radar  LOS  is 
unobstructed;  low  signifies  that  the  radar 
LOS  is  obstructed);  the  solid  curve  below  is 
a  plot  of  the  error  in  estimating  fixed-wing 
position  with  just  the  radar;  the  dashed  curve 
is  a  plot  of  the  error  that  results  from  using 
both  the  radar  and  the  passive  sensors.  Note 
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that  sensor  fusion  tends  to  Improve  estimates 
of  target  location,  especially  when  the  radar 
LOS  is  obstructed. 


•  SIGNIFICANT  IMPROVEMENT  IN  ESTIMATING  SEARING.  ON  AVERAGE; 

6.5*  (FUSED)  VERSUS  34.8*  (IRST  ONLY) 

•  EXTENDED  AMOUNT  OF  TIME  DURING  WHICH  TARGET'S  BEARING  CAN  BE  TRACKED 
FROM  546  TO  832  S 

•  IMPROVED  BEARING  PREDICTIONS  FOR  PERIODS  OF  NO  RADAR  REPORTS  FACILITATE 
HFACQUIRING  TARGET 


Figure  4.  Fused  Bearing  Tracking  on 
Helicopter  (Sensor  Site  3) 


- —  estimated  position  error  (METERS)  WITH  RADAR  ONLY 

_ _ _  estimated  position  error  (metehs)  after  adoino  passive  sensor 

BOTH  targets  viewed  from  sensor  site  3 


•  LOS  TO  FIXED  WING  IS  OBSTRUCTED 
DURING  22-S  TO  40-S  INTERVAL 

•  BEARING  FROM  PASSIVE  SENSORS 
IMPROVES  ESTIMATES  OF  3-D  POSITION  BY 
ABOUT  40%  OVER  THIS  INTERVAL 


•  LOS  TO  HELICOPTER  IS  OBSTRUCTED 
DURING  300-S  TO  400-S  INTERVAL 

•  BEARING  FROM  PASSIVE  SENSORS 
IMPROVES  ESTIMATES  OF  3-D  POSITION 
BY  ABOUT  57%  OVER  ENTIRE  FLIGHT  PATH 


Figure  5.  3-D  Tracking  for  Fixed  Wing 

and  Helicopter  Targets 


Note  the  relative  improvement  (%)  due  to 
fusion  in  estimating  target  position  shown 
in  the  right  column. 


Table  2.  Position  Estimation  for 
Helicopter  Example 


Radar  + 

Radar 

Passive 

Percent 

Only 

Sensors 

Improvement 

LOS  unobstructed 

60 

39 

35 

error  (m) 

LOS  obstructed 

A74 

188 

60 

error  (m) 

Total  error  (m) 

263 

112 

57 

The  multi-sensor  tracker  simulation  re¬ 
sults  as  shown  in  Figure  4  indicate  that,  on 
the  average,  the  multi— sensor  tracker  tracked 
the  manueverlng  helicopter  within  1.5  degrees 
from  the  truth  for  a  total  of  832  seconds 
about  92%  of  the  time  period  of  904  seconds 
in  which  the  Hind-D  was  within  the  sensors' 
detection  range.  For  the  IRST  only,  the  cor¬ 
responding  numbers  are  1.2  degrees,  546  sec¬ 
onds,  68%,  and  804  seconds,  respectively.  Dur¬ 
ing  periods  of  no  reports,  the  smaller  predic¬ 
tion  errors  from  the  multi— sensor  tracker  sig¬ 
nificantly  facilitate  target  reacqulsltion . 


Bearing-Only  Evaluation  Analysis 

The  merits  of  multi-sensor  tracking  versus 
single-sensor  tracking  from  the  IRST,  NI ,  and 
RFDF  were  evaluated  in  terms  of  bearing  error 
against  the  known  grpundtruth.  As  shown  for 
the  Hind-D  target  In  Table  3,  the  multi-sensor 
tracker  had  a  total  of  833  reports  and  44  pre¬ 
dictions.  The  individual  sensors  had  fewer 
reports,  ranging  from  762  for  the  NI  to  178 
for  the  RFDF  and  had  to  make  more  predictions 
(as  many  as  202  for  the  IRST).  These  predic¬ 
tions  are  necessary  when  no  reports  are  avail¬ 
able  from  any  sensors  and  the  3-D  tracker 
requires  a  bearing  reading. 


Similar  results  for  tracking  a  helicopter 
appear  in  Figure  5(B):  the  dotted  curve  is  a 
plot  of  the  error  that  results  from  using  both 
the  radar  and  the  passive  sensors  (fusion). 
Note  that  sensor  fusion  generally  Improves 
estimates  of  target  location.  In  some  of  the 
cases,  the  fused  performance  is  actually  worse 
(e.g.,  t  =  900  seconds,  t  =  960  seconds);  this 
is  due  to  fused  bearing  tracker  errors  during 
target  manuevers  and  is  mostly  a  result  of  the 
NI  sensor,  the  least  accurate  sensor  in  the 
suite.  A  number  of  manue ver-adaptl ve  ap¬ 
proaches  could  be  used  to  solve  these  spe¬ 
cific  problems. 

The  helicopter  results  are  further  sum¬ 
marized  in  Table  2.  Two  conditions  are  repre¬ 
sented;  the  radar  LOS  unobstructed  and  the 
radar  LOS  obstructed.  When  the  radar  Is  off, 
the  tracker  continues  to  estimate  target  posi¬ 
tion  with  the  passive  sensors  (fusion),  or 
without  the  passive  sensors  (if  not  avail¬ 
able).  The  left  column  shows  the  average 
position  error  per  radar  update  (4  seconds) 
for  both  conditions  with  the  radar  only  (l.e., 
no  passive  sensors).  The  middle  column  shows 
the  error  when  the  radar  measurements  are 
fused  with  the  passive  sensor  measurements. 


Table  3.  Bearing  Tracker  Performance 
Comparison  for  Helicopter 
(average  error  In  degrees) 


IRST 

NI 

RFDF 

All  +  Radar 

Observed 

1.2 

1.9 

1  .4 

1.5 

(number  of 
contacts) 

(316) 

(762) 

(178) 

(833) 

Prediction 

55  .7 

96.2 

106.9 

28.2 

(number  of 
predictions ) 

(  202) 

(49) 

(139) 

(44) 

Total 

34  .8 

23.7 

70.8 

6.5 

( t  rack  er 
operations) 

(518) 

(811) 

(317) 

(877  ) 

SUMMARY 

This  paper  describes  a  sensor  fusion  sim¬ 
ulation  system  and  focuses  on  the  merits  of 
multi-sensor  tracking.  The  system  configura¬ 
tion  chosen  is  based  on  a  simple  alpha— beta 
tracking  approach.  In  this  paper,  we  are  pri¬ 
marily  concerned  with  the  tracking  improve¬ 
ments  that  could  result  from  fusing  passive 
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sensors  with  a  radar  in  a  ground-to-air  sce¬ 
nario.  Our  results  Indicate  that  Intermediate 
angle  Information  can,  on  the  average,  Improve 
a  3-D  radar’s  prediction  of  target  location 
by  more  than  a  factor  of  two. 


ACKNOWLEDGMENTS 

We  would  like  to  acknowledge  Dave  Kabel 
for  his  technical  contributions  to  this  study. 
We  would  also  like  to  thank  Beverly  Littlejohn 
for  her  assistance  In  preparing  this  manu¬ 
script. 


REFERENCES 

1.  Automation  In  Combat  Aircraft,  Nation¬ 
al  Academy  Press,  Washington,  D.C.  1982. 

2.  Benedict,  T.R.  and  Bor  dner ,  G ,  W.^(  1 96  2  ); 
"Synthesis  of  an  optimal  set  of  track-while- 
scan  smoothing  equations,"  IRE  Trans,  on  Auto- 


mat.  Control,  Vol. 

AC-7,  No.  4, 

(Jul),  pp 

27-32. 

3.  Bhagavan,  B. 

and  Polge, 

R.  (1974) 

"Performance  of  g-h  filter  for  tracking  man- 
ueverlng  targets,"  IEEE  Trans,  on  Aerosp.  and 
Electron.  Sys.,  Vol.  AES-10,  No.  6,  (Nov), 
pp .  864-866  . 


4.  Blackman,  S.S.  (1986);  Multiple  Tar¬ 
get  Tracking  with  Radar  Applications,  Artech 
House,  Dedham,  MA. 

5.  Gelb,  A.,  (1974);  Applied  Optimal  Esti¬ 
mation,  Cambridge,  MA:  MIT  Press. 

6.  Greer,  T.H,  (1985);  "Artificial  In¬ 
telligence:  a  new  dimension  in  EW,"  Defense 
Electronics  (Oct),  pp.  108-128. 

7.  Quigley,  A.  (1973),  "Tracking  and  as¬ 
sociated  problems",  IEEE  Conf.  on  Radar  - 
Present  and  Future,  London,  (Oct),  pp.  352- 


8.  Singer,  R.A.  and  Benke ,  K,  (1971), 
"Real-time  tracking  filter  evaluation  and  se¬ 
lection  for  tactical  applications,"  IEEE 
Trans,  on  Aerospace  and  Electronic  Sys.,  Vol. 
AES-7,  No.  1,  (Jan),  pp.  100-110. 

9.  Trunk,  G.V.  and  Wilson,  J.D.  (1987), 
'Association  of  DF  Bearing  Measurements  with 
Radar  Tracks, "  IEEE  Trans,  on  Aerospace  and 
Electronic  Systems,  Vol.  AES-23,  No.  4,  (Jul). 
pp.  438-447. 


200 


ROBUST  FUSION  OF  LOCATION 
INFORMATION 

Raymond  McKendall  and  Max  Mintz 


Department  of  Systems  Engineering 
and 

Department  of  Computer  and  Information  Science 
University  of  Pennsylvania 
Philadelphia,  PA  19104-6389 


Abstract 


The  purpose  of  this  paper  is  to  examine  a  sensor  fusion  problem  for  location  data  using 
statistical  decision  theory  (SDT).  The  contribution  of  this  paper  is  the  application  of 
SDT  to  obtain:  (i)  a  robust  test  of  the  hypothesis  that  data  from  different  sensors  is 
consistent;  and  (ii)  a  robust  procedure  for  combining  the  data  which  pass  this  preliminary 
consistency  test.  Here,  robustness  refers  to  the  statistical  effectiveness  of  the  decision 
rules  when  the  probability  distributions  of  the  observation  noise  and  the  a  priori  position 
information  associated  with  the  individual  sensors  are  uncertain.  Location  data  refers 
to  observations  of  the  form:  Z  =  0 V,  where  V  represents  additive  sensor  noise  and  0 
denotes  the  “sensed”  parameter  of  interest  to  the  observer.  While  the  theory  developed 
in  this  paper  applies  to  several  uncertainty  classes,  the  focus  of  this  presentation  is  on 
e-contamination  models,  which  allow  one  to  account  for  heavy-tailed  deviations  from 
nominal  sampling  distributions. 
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1  Introduction 

The  purpose  of  this  paper  is  to  examine  a  sensor  fusion  problem 
for  location  data  using  statistical  decision  theory  (SDT).  The 
contribution  of  this  paper  is  the  application  of  SDT  to  obtain; 
(i)  a  robust  test  of  the  hypothesis  that  data  from  different  sensors 
is  consistent;  and  (ii)  a  robust  procedure  for  combining  the  data 
which  pass  this  preliminary  consistency  test.  Here,  robustness 
refers  to  the  statistical  effectiveness  of  the  decision  rules  when 
the  probability  distributions  of  the  observation  noise  and  the  a 
priori  position  information  associated  with  the  individual  sensors 
are  uncertain.  Location  data  refers  to  observations  of  the  form: 
Z  —  B  •\-'V  ^  where  Y  represents  additive  sensor  noise  and  B  de¬ 
notes  the  “sensed”  parameter  of  interest  to  the  observer.  The 
parameter  B  is  referred  to  as  a  location  parameter,  since  the  dis¬ 
tribution  of  Z  is  obtained  from  the  distribution  of  V"  by  a  trans¬ 
lation.  This  paper  does  not  focus  on  a  specific  robotics  problem, 
but  seeks  instead  to  abstract  the  essential  components  of  a  family 
of  such  data  fusion  problems  and  obtain  a  corresponding  family 
of  solutions.  While  the  location  parameter  fusion  problem  is  only 
one  of  many  possible  fusion  paradigms,  it  does  provide  a  useful 
starting  point  for  considering  more  complicated  problems,  e.g., 
nonlinear  sensor  models  of  the  form:  Z  =  h{B)  -\-  V . 

The  fusion  of  location  data  for  location  estimation  has  been 
discussed  in  the  robotics  literature  by  Ayache  and  Faugeras  (1987), 
Chatila  and  Laumond  (1985),  Durrant-Whyte  (1986a,  1986b), 
Matthies  and  Shafer  (1986),  and  Smith  et  al  (1986,  1987).  In 
Ayache  and  Faugeras  (1987),  Chatila  and  Laumond  (1985),  and 
Smith  et  al  (1986,  1987),  the  authors  assume  the  sensor  noise 
can  either  be  adequately  modeled  by  Gaussian  distributions  with 
known  means  and  covariances,  or  by  distributions  characterized 
only  by  specified  first  and  second  moments.  In  this  latter  sit¬ 
uation  the  analysis  is  limited  to  affine  decision  rules  which  are 
evaluated  on  the  basis  of  quadratic  loss.  Subject  to  these  limi¬ 
tations  (affine  procedures  and  quadratic  loss),  the  results  of  this 
decision  model  are  “equivalent”  to  invoking  a  Gaussian  model. 
Whereas,  in  Durrant-Whyte  (i986a,  1986b)  and  Matthies  and 
Shafer  (1986),  the  authors  recognize  the  need  to  address  the  ex¬ 
istence  of  non-Gaussian  sampling  distributions.  While  Gaussian 
models  offer  a  degree  of  mathematical  elegance  and  simplicity,  the 
adoption  of  Gaussian  models  for  sampling  distributions  imposes 


substantial  risk  on  the  decision-maker  when  the  actual  sampling 
distributions  possess  heavy  tails,  e.g.,  they  exhibit  departures 
from  the  Gaussian  model  in  the  form  of  e-contamination  uncer¬ 
tainty  classes.  This  lack  of  robustness  of  decision  procedures 
based  on  Gaussian  sampling  distributions  has  been  discussed  in 
the  statistics  literature  for  more  than  thirty  years.  Detailed  ex¬ 
aminations  of  the  theory  and  applications  of  robust  statistical 
inference  appear  in  the  monographs  by  Huber  (1981)  and  Ham¬ 
pel  et  al  (1986). 

In  the  sequel  we:  (i)  delineate  several  paradigms  for  robust 
fusion  of  multi-sensor  location  data;  (ii)  introduce  some  essential 
nomenclature  and  definitions  from  SDT;  (iii)  review  the  earlier 
decision-theoretic  results  on  which  this  paper  is  based;  and  (iv) 
present  and  discuss  a  methodology  for  robust  fusion  of  multi¬ 
sensor  location  data. 

Our  presentation  emphasizes  the  statement  and  application 
of  the  relevant  theory.  Proofs  of  theorems  are  omitted.  The 
reader  is  referred  to  journal  articles  and  reports  for  these  details. 

2  Paradigms  for  Sensor  Fusion  of  Loca¬ 
tion  Data 

In  this  section  we  delineate  several  paradigms  for  robust  fusion 
of  location  data.  We  restrict  our  attention  to  observations  of 
one-dimensional  location  parameters.  The  results  of  this  one¬ 
dimensional  analysis  can  be  applied  to  the  multidimensional  case 
by  doing  a  component  by  component  analysis.  Alternatively,  one 
can  pursue  a  formal  multidimensional  extension  of  the  method¬ 
ology  presented  in  this  paper. 

The  general  one-dimensional  paradigm  is  delineated  as  fol¬ 
lows.  We  assume  that  we  are  given  the  sampled  outputs  of  r 
sensor  systems  {5,-  :  1  <  i  <  r}.  We  denote  the  sampled 
output  of  1  <  A:  <  A,-  by: 

Zik  =  fJ^i  -}-  Wi  4-  +  Kfcj  (2-1) 

where: 

•  o-i  <Gi<  hi,  denotes  an  unknown  location  parameter  with 
known  bounds  a/  and  bi,  (Note:  the  bounds  a.-  and  5,-  may 
assume  infinite  values.)  In  many  applications  there  is  a 
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common  interval  of  location  parameter  uncertainty  for  all 
sensors.  However,  there  is  no  need  to  make  this  assumption 
in  the  following  mathematical  developments. 

•  /X,-,  denotes  a  known  constant  (offset)  associated  with  the 
position  of  sensor  Si  with  respect  to  a  common  origin. 

•  Vik,  denotes  the  additive  observation  noise  associated  with 
the  observation  (sample)  from  5,-.  The  random  variables 

•  1  <  ^  <  ^i}  assumed  to  be  independent  and 
identically  distributed  (i.i.d.).  We  further  assume  that  the 
noise  process  associated  with  Si  is  independent  of  the  noise 
process  associated  with  when  i  ^  j.  Finally,  we  assume 
that  the  probability  distribution  of  Vik  belongs  to  a  given 
uncertainty  class  of  distributions,  We  do  not  assume 
that  the  noise  processes  associated  with  different  sensors 
are  identically  distributed. 

•  Wi,  denotes  the  uncertainty  in  the  position  of  sensor  Si  with 

respect  to  a  common  origin.  We  consider  two  cases:  (i)  the 
position  uncertainty  of  Si  can  be  expressed  by  a  known  in¬ 
terval  [li,ui]  —  with  no  a  priori  probabilistic  description; 
or  (ii)  the  position  uncertainty  of  5,-  can  be  expressed  by 
an  unknown  probability  distribution  from  a  given  uncer¬ 
tainty  class  Vi.  In  each  case,  we  assume  that  the  position 
uncertainty  of  Si  is  independent  of  the  observation  noise 
{Vik  -  <  k  <  Ni},  and  independent  of  the  observation 

noise  and  position  uncertainty  of  the  other  sensors. 

Remark  2.1  Without  loss  of  generality,  we  can  assume  that  the 
known  offsets  {fii  :1  <i  <t)  are  each  zero,  since  nonzero  values 
can  be  subtracted  from  the  observations  {Zik}-  Further,  if  the 
known,  generally  asymmetric,  interval  of  uncertainty  [a^,  6,]  in 
8i  is  finite,  then  the  observations  {Zik)  shifted  and  the 

interval  of  uncertainty  [a,  ,  6,]  can  be  replaced  by  [— d,-,  di],  where 
di  ~  {hi  -  a,)/2.  Similarly,  we  can  assume  the  interval  of  sen¬ 
sor  position  uncertainty  (where  applicable)  is  again  symmetric. 
Thus,  (2.1)  can  be  replaced  by: 

Zik  =  Wi^-ei-^Vik,  (2.2) 

where;  1  |  <  d,-,  and  (where  applicable)  ]  W,-  |  <  T/i,  1  <  t  <  r. 

The  uncertainty  classes  Ti  and  (where  applicable)  "P,-,  1  <  i  < 
r,  denote  neighborhoods  in  the  space  of  probability  distributions 
which  are  deemed  to  adequately  characterize  the  uncertainty  in 
the  specifications  of  the  sampling  distributions.  Models  for  sev¬ 
eral  uncertainty  classes  are  described  in  Sections  4,  5,  and  6. 

As  stated  in  the  introduction,  the  purpose  of  this  paper  is  to 
examine  a  sensor  fusion  problem  for  location  information  using 
SDT.  The  contribution  of  this  paper  is  the  application  of  SDT 
to  obtain:  (i)  a  robust  test  of  the  hypothesis  that  data  from 
different  sensors  is  consistent,  i.e.,  testing  the  hypothesis  that 

=  ^j,  1  <  t  <  j  <  r;  and  (ii)  a  robust  procedure  for  combining 
the  data  which  pass  this  preliminary  consistency  test.  Again, 
robustness  refers  to  the  statistical  effectiveness  of  the  decision 
rules  when  the  probability  distributions  of  the  observation  noise 
and  the  a  priori  position  information  of  the  individual  sensors  are 
uncertain. 

In  the  following  section,  we  introduce  the  notions  of  robust 
minimax  decision  rules  and  robust  confidence  procedures.  These 
concepts  provide  the  basis  for  the  developments  in  the  remainder 
of  this  paper. 

3  Nomenclature  and  Definitions 
from  SDT 

The  standard  statement  of  a  minimax  location  parameter  esti¬ 
mation  problem  includes  as  given:  a  parameter  space  a  space 
of  actions  A\  a  loss  function  L  defined  on  >1  x  12;  and  a  CDF 
jF.  If  the  underlying  CDF  is  imprecisely  known,  then  this  stan¬ 
dard  minimax  decision  model  must  be  reformulated  to  account 
for  this  additional  uncertainty.  Statistical  decision  rules  which 
are  applicable  in  this  more  general  problem  setting  are  referred 
to  as  robust  procedures. 


This  paper  considers  robust  fixed  size  confidence  procedures 
for  a  restricted  parameter  space.  These  robust  confidence  pro¬ 
cedures  are  based,  in  turn,  on  the  solution  of  a  related  robust 
minimax  decision  problem: 

Let  Z  denote  a  vector  of  N  i.i.d.  observations  of  a  scalar 
random  variable  with  CDF  F{z  -  ^),  where  F  e  F,  a.  ^ven  un¬ 
certainty  class.  Let  Q  —  A  =  [— d,  d],  and  define  a  zero-one  loss 
function  i  on  ^  X  12: 

where  e  >  0,  is  given.  Further,  let  R(6,  9,  F)  =  E[L{6^  9)  \  9,  F] 
denote  the  risk  function  of  the  decision  rule  S  given  ^  G  12  and 
FeF. 

Definition  3.1  An  estimator  S*  is  said  to  be  a  robust  minimax 
estimator  for  9,  if  for  all  S: 

sup  9,  F)  <  sup  R(S^  9,  F). 
een  eeii 

Based  on  these  definitions  and  assumptions,  we  seek  a  robust 
minimax  estimator  6*  for  9.  For  brevity,  we  restrict  our  consid¬ 
eration  to  the  case  when  d/e  is  an  integer  >  2. 

Observation  3.1  The  connection  between  the  robust  minimax 
rule  6*{Z)  and  a  robust  fixed  size  confidence  procedure  is  ob¬ 
tained  by  noting  that: 

C*iZ}  =  [<5"(Z)  -  c,  ^"(Z)  -f  e] 

can  be  interpreted  as  a  robust  confidence  procedure  of  size  2e 
which  has  the  highest  confidence  coefficient  inf^^F  Fe,F[^  G  C'*(Z)]. 

Sections  4,  5,  and  6  of  this  paper  are  organized  as  follows: 

Section  4  reviews  the  solution  to  the  related  minimax  es¬ 
timation  problem  where  F  =  A/"(0,(7^),  and  a  is  given.  These 
results  provide  the  basis  for  the  solution  to  the  robust  minimax 
estimation  problem  where  F  =  {F  —  Af{0ya'^):  a  <  0-^}. 

Section  5  extends  these  robust  minimax  results  to  uncer¬ 
tainty  classes  which  contain  non-Gaussian,  asymmetric,  and  dis¬ 
continuous  CDF’s.  Robust  median-minimax  rules  are  introduced 
and  evaluated. 

Section  6  develops  a  theory  and  methodology  for  robust  sen¬ 
sor  fusion  of  location  information  based  on  the  theory  presented 
in  Sections  4  and  5. 

4  Robust  Minimax  Rules  and 
Gaussian  Uncertainty  Classes 

4.1  Preliminary  Minimax  Results 

Zeytinoglu  and  Mintz  (1984)  addresses  the  related  minimax  esti¬ 
mation  problem  where  F  =  ^^(0,^^),  and  a  is  given.  The  main 
result  requires  Definition  4,1  and  is  summarized  by  Theorem  4.1: 
Definition  4.1  Let  C  denote  the  class  of  nonrandomized,  odd, 
monotone  nondecreasing  decision  rules  A.  Let  A  cC 


denote  the  set  of  rules  ^(t),  defined  for  2  >  0  by: 

'  d~  €, 

c  -t-  On  +  2ne  <  t\ 

«(«)  =  . 

t  —  0.2 1 

c  -f  02  +  2e  <  t  <  c  -f  02  +  4c; 

(4.1) 

2e  -f  c, 

c  -f  Cl  +  2e  <t<  c  -f-  a'2  +  2e; 

f  -  ai. 

c-foi  <t<c-fai-f  2c; 

c, 

c  <t<  c  -f  Oi ; 

t, 

0  <  t  <  c; 

where:  0  < 

ai  <  02 

<  . . .  <  a„  <  oo,  d  =  (2n  -f-  l)e  -f  c,  and 

c  equals  zero  (e)  if  d  is  an  odd  (even)  multiple  of  e.  (Note:  due 
to  the  existing  symmetry,  all  function  definitions  are  stated  for 
nonnegative  arguments.) 
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(4.6) 


Theorem  4.1  Let  L  denote  the  loss  function  (3.1).  \l  Z  ^ 
A/'(^,cr^),  where  a  is  given,  then  for  any  iV  >  1  there  exists  a 
(globally)  minimax  admissible  rule  5*  e  A  which  is  Bayes  with 
respect  to  a  least  favorable  prior  distribution  A*.  Further,  6* 
depends  on  Z  through  the  sample  mean  Zjq. 

Proof:  See  Zeytinoglu  and  Mintz  (1984). 

Observation  4.1  (Zeytinoglu  and  Mintz,  1984)  U  N  -  1,  6  e 
A,  and  F  is  any  continuous  CDF  which  is  symmetric  about  zero, 
then: 

'  F(a„-e),  d-2e<  0  <d; 

F(a„_i  -  e),  0  =  d-2e; 

F(~o„  -  c)  +  F(art_i  -  e),  d  —  4c  <  0  <d~2e; 


R{S,  0,  F) 


f’(_a2-e)+F(ai~c), 
F{-a2  ~e)  +  {c/e)F{~e) 
+(l™c/e)F(-ai-e), 
F(~ai  -e)  +  {c/e)F{~e) 
+(l-c/c)F(-ai-c), 
2F(-ai-e). 


c+c<  0  <c  +  3e; 


0  z=  c+  e; 


0  <  0  <  c  +  e; 

^  =0. 

(4.2) 


Remark  4.1  R(6^  F),  F),  and  A*  have  the  following 

characteristics: 


•  If  ^  e  A,  then  i2(^,  F)  is  a  piecewise  constant  function 
of  9  over  the  sets  of  a  finite  partition  of  D. 

•  The  minimax  rules  S*  are  “almost”  equalizer  rules,  in  the 
sense  that  the  nondegenerate  piecewise  constant  segments 
of  the  risk  function  are  equalized  at  the  minimax  risk  M. 

•  The  risk  expression  (4.2)  can  be  readily  modified  to  include 
CDF’s  F  which  are  both  asymmetric  and  discontinuous. 
The  generalized  (asymmetric)  risk  function  R{6,  F)  is 
again  a  piecewise  constant  function  of  9  over  the  sets  of  the 
finite  partition  of  Q.  expressed  in  (4.2). 

•  A*  is  defined  by  a  density  function  which  is  symmetric  and 
piecewise  constant. 


4.2  Robust  Minimax  Rules 


Definition  4.2  Let: 


T  =  {F  ~  a  <  (T^},  (4.3) 

denote  an  uncertainty  class  of  Gaussian  distributions. 

Definition  4.3  The  CDF  F^  =  A/’(0,cr2)  defines  the  upper- 
envelope  of  T  (4.3)  in  the  sense  that:  F(x)  <  jP„(a:)  for  all  G 
and  X  <  0. 


The  following  theorem,  which  is  the  main  result  of  this  section, 
extends  the  results  of  Theorem  4.1  to  the  robust  minimax  esti¬ 
mation  problem. 

Theorem  4.2  Let  F  denote  the  uncertainty  class  (4.3)  with 
upper- envelope  Fh  =  A^(0,(Ty).  Let  h*  denote  the  minimax  rule 
obtained  through  Theorem  4.1  based  on  a  sample  size  IV  and 
CDF  Fh.  There  exists  abound  .B(d/e,  iV,  Fh),  such  that  if  e  >  jB, 
then  ^ *  is  a  robust  minimax  admissible  Bayes  rule  in  the  sense  of 
Definition  3.1.  Further,  depends  on  Z  through 
Proof:  See  Zeytinoglu  and  Mintz  (1988). 

Example  4.1  Let  d  =  3e,  e  -  0.1,  N  =  49,  =  2,  Fh  = 

A^(0»  andG  =  A/'(0,cr2/F/’).  Applying  Theorem  4.2,  the  min¬ 
imax  rule  6*{Zj^)  and  risk  function  R{8*  9,  Fh)  corresponding  to 
Fh  are: 


6*{Zn)  - 


R{6*  9, 


^e, 

Zn 

0, 


oi  4-  2e  <  Zjv; 


ai, 

£  Zfi  < 

ax  -{•  2e; 

(4.4) 

0  ^  Zn  < 

G(a] 

V 

1 

<  3c; 

Gi~ 

ax  -  c),  9 

=  e; 

(4.5) 

2G{- 

-ax  -  e),  0  <  ^ 

< 

where  ai  satisfies: 


G{ai  —  e)  =  2G{—ai  —  e). 

In  this  example  ai  —  0.092,  and  the  corresponding  minimax  risk 
is  0.49.  The  bound  B,  which  is  derived  in  Zeytinoglu  and  Mintz 
(1988),  is: 

B{d/e,  N,  f„)  =  -(l/2VF)f-'(l/4)  =  0.0966. 

Here,  c  >  B,  and  thus  6*  (4.4)  is  a  robust  minimax  rule. 

5  Robust  Minimax  Rules  and 

Non-Gaussian  Uncertainty  Classes 

5.1  Preliminaries 

This  section  extends  the  robust  minimax  results  of  Section  4  to 
uncertainty  classes  which  contain  non-Gaussian,  asymmetric,  and 
discontinuous  CDF’s. 

Definition  5.1  Let  F  denote  an  uncertainty  class  with  upper- 
envelope  F^i 

:  f(i-)  <  I  <  0;  and  F{x)  >  F^(x),  x  >  0}, 

(5.1) 

where  Fh  has  a  density  function  which  is  unimodal  and  symmetric 
about  zero.  (Note:  F’(a:“)  denotes  the  left-hand  limit.) 

Remark  6.1  We  allow  Fh  to  be  substochastic,  i.e.,  Fh  can  have 
less  than  unit  probability  mass.  Thus,  the  usual  e-contamination 
models  can  be  represented  by  F  (5.1). 

The  main  results  of  this  section  are  based  on  the  following  theo¬ 
rem  which  addresses  the  existence  and  construction  of  C-minimax 
and  minimax  rules  for  single- sample  decision  problems.  (Note: 
A  rule  is  (robust)  X>-imnimax  if  it  is  (robust)  minimax  within  the 
class  V,  A  rule  is  P-Bayes  if  it  is  Bayes  within  the  class  V.  A 
rule  is  D-admissible  if  it  is  admissible  within  the  class  X>.) 

Theorem  5.1  Let  iV  =  1.  If  the  CDF  F  has  a  density  function 
which  is  unimodal  and  symmetric  about  zero,  then  there  exists  a 
C-minimax  rule  G  A.  Further,  if  F  possesses  a  (strictly)  mono¬ 
tone  likelihood  ratio,  then  8*  is  a  minimax  (admissible)  Bayes 
rule. 

Proof:  See  Zeytinoglu  and  Mintz  (1984). 

5.2  The  Single-Sample  Case 

The  following  theorem  extends  the  results  of  Theorem  5.1  to  the 
single-sample  robust  C-minimax  estimation  problem. 

Theorem  5.2  Let  N  —  \^F  denote  the  uncertainty  class  (5.1) 
with  upper-envelope  Fh,  and  8*  denote  the  C-minimax  rule  ob¬ 
tained  through  Theorem  5.1  based  on  CDF  Fh.  There  exists  a 
bound  B(d/e,  A'  =  1,  F;),  such  that  if  e  >  B,  then  ^*is  a  robust 
C-minimax  rule.  Further,  if  Fh  possesses  a  (strictly)  monotone 
likelihood  ratio,  then  8*  is  a  robust  minimax  (admissible)  Bayes 
rule. 

Proof:  See  Zeytinoglu  and  Mintz  (1988). 

Example  5.1  [an  €-contamination  model]  Let  d  =  3e,  and  F 
denote  the  uncertainty  class: 

.F=  {A:  B  =  (1  -  €)$  4- eB},  (5.2) 

where:  $  =  A/"(0,1),  the  CDF  U  is  symmetric  about  zero,  and 
0  <  f  <  1/2.  The  corresponding  (substochastic)  upper-envelope 
is: 

Fh  =  (1  -  e)$  -t-  c/2.  (5.3) 

In  this  example  B  is: 

B(d/e,  A  =  1,  F’,,)  =  -(l/2)F“i(l/4).  (5.4) 

Applying  Theorem  5.2,  the  C-minimax  rule  8*{Z)  and  risk  func¬ 
tion  R(8*  9,  Fu)  corresponding  to  Fh  are: 
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f  2e 

,  ai  -k  2€  <  Z; 

(5.5) 

5*(Z)={  Z 

—  «!,  <*1  ^  ^  d"  2^; 

1  0- 

0  <  Z  <  ai; 

f  F„(ai  -  c),  €  <B  <  3c; 

(5.6) 

e,  K)  = 

<  Fu(-ai  “  c),  B=  c; 

[  2Fu(-ai  -  c),  0  <  d  <  c; 

where  ai  satisfies: 

Ftt(ai  —  c)  —  2Ftt(— ui  —  e), 

(5.7) 

or  equivalently, 

$(ai  - 

e)  =  2$(-o, -e)  +  f/2(l-€). 

(5.8) 

Thus,  if  e  >  B,  then  S*  (6.5)  is  a  robust  C-minimax  rule  for  this 
e-contamination  model. 

This  solution  is  easily  extended  to  other  values  of  d/e  and 
nominal  distributions.  The  required  calculations  include:  the 
computation  of  the  vector  a  which  parameterizes  the  underlying 
C-minimax  rule  and  the  computation  of  the  bound  B(d/c,  N  = 

_  which  are  each  readily  obtained  by  means  of  a  Newton- 
Raphson  algorithm. 


In  this  example  ai  -  e  =  0.2133,  and  the  corresponding  C- 
median-minimax  risk  is  0.50. 

Theorem  5.4  Let  N  >  I,  T  denote  the  uncertainty  class  (5.1) 
with  upper-envelope  and  S*  denote  the  C-median-minimax 
rule  obtained  through  Theorem  5.3  based  on  CDF  There 
exists  a  bound  F(d/e,  Nj  Fu),  such  that  if  e  >  F,  then  6  is 
a  robust  C-median-minimax  rule.  Further,  if  the  upper-envelope 
CDF  of  possesses  a  (strictly)  monotone  likelihood  ratio, 

then  ^•‘is  a  robust  median-minimax  (median-admissible)  median- 
Bayes  rule. 

Proof:  See  Zeytino^u  and  Mintz  (1988). 

Example  6.3  [Example  5.2  revisited  and  extended]  Let  d  =  3e, 
e  =  0.2133,  iV  =  3,  F„  denote  the  CDF  of  the  double  expo¬ 
nential  distribution  (5.9),  and  F  denote  (5.1).  In  this  example 
B(d/e,  N,  Fu)  is: 

B(d/e,  N,  Fu)  =  -(l/2)G-\l/4)  =  e,  (5.13) 

where:  G  =  F^(3  -  2Fu).  Since  e  =  F,  it  foUows  from  Theo¬ 
rem  5.4  that  S*  (5.10)  is  a  robust  C-median-minimax  rule  with 
corresponding  maximum  risk  0.50. 


5.3  The  Multi-Sample  Case 

This  section  extends  the  robust  C-minimax  results  of  Theorem 
5.2  to  the  multi-sample  problem  by  restricting  the  class  of  esti¬ 
mators  to  rules  of  the  form  ^(!r(Z)),  where:  S  ECf  T\E^  — ►  E\ 
and  r(Z)  possesses  a  density  function  which  is  unimodal  and 
symmetric  about  6.  Examples  of  candidate  T  statistics  include: 
the  sample  mean,  the  sample  median,  and  other  symmetric  linear 
combinations  of  order  statistics.  In  the  remainder  of  this  section 
we  consider  the  sample  median.  See  Tieytinoglu  and  Mintz  (1988) 
for  a  comparison  of  the  performance  of  these  restricted  decision 
rules  with  an  alternative  —  the  highest  posterior  density  (HPD) 
credible  set,  Berger  (1985). 

Definition  5.2  Let  Zm  denote  the  median  of  the  JV  observar 
tions  Z.  The  decision  rule  ^*(Za/),  defined  by  the  composition 
8*  o  ZAf ,  is  said  to  be  a  median-minimax  estimator  for  if  8* 
is  a  minimax  rule  in  the  usual  sense.  The  respective  definitions 
of  robust  median-minimax  rules,  C-median-minimax  rules,  and 
robust  C-median-minimax  rules  are  obtained  as  before.  (Note:  H 
N  is  even,  Zm  =  (^[JV/2]  +  ^[(Ar/2)+i])/2-) 

Theorem  5.3  Let  iV  >  1.  If  the  CDF  F  has  a  density  function 
which  is  unimodal  and  symmetric  about  zero,  then  there  exists 
a  C-median-minimax  rule  G  A.  Further,  if  the  CDF  of  {Zm  ~ 
6)  possesses  a  (strictly)  monotone  likelihood  ratio,  then  8*  is  a 
median-minimax  (median-admissible)  median- Bayes  rule. 

Proof:  See  Zeytinogju  and  Mintz  (1988). 


Example  5.2  Let  d  =  3c,  e  =  0.2133,  JNT  =  3,  and  F  denote  the 
CDF  of  the  double  exponential  distribution: 


F(x)  =  1 

1  -  (l/2)exp(~«),  a;  >  0; 

(1/2)  exp  (x),  x<0. 

(5.9) 

Let  F'  denote  the  CDF  of  the  centered  sample  median  Zm  -B  = 
r(Z)  -  B.  In  this  example  F'(t)  =  F*(t)(3  -  2F(f)).  Applying 
Theorem  5.3,  the  C-median-minimax  rule  6*{T)  and  risk  function 
R{6*  B,  F)  are: 

(  2c, 
^*(r)={  T- 

l 

oi  -1-  2e  <  T; 

■  »  oi  <  r  <  oi  -t*  2e; 

0  <  T  <  ai; 

(5.10) 

B,F) 

f  F'{ai  -  c),  e  <B  <  3c; 

[  F'(-ai  -  c),  B=  c; 

[  2F'(-ai  -  c),  Q<B<€; 

(5.11) 

where  a\  satisfies: 

f'(oi-e)  =  2F'(-oi-e). 

(5.12) 

5.4  Two  Special  Cases 

There  are  two  limiting  cases  which  are  worthy  of  special  mention: 

Case  1.  (d  =  2e)  If  d  =  2c,  then  Theorems  4.2,  5.2  and  5.4 
apply  for  all  c  >  0,  i.e,,  in  each  instance  F  is  zero.  The  ro^st 
rules  obtained  in  Theorems  4.2, 5.2,  and  5.4  are  respectively:  Zfj, 
Z,  and  Zm  truncated  to  [-e,  e],  with  respective  minimax  risks: 
Fu(“-v'iVc),  F„(-c),  and  (7u(-e),  where  denotes  the  CDF  of 
the  median  based  on  F^. 

Case  2.  (d  oo)  If  Q  =  F^,  then  Theorems  4.2,  5.2  and 

5.4  apply  for  all  c  >  0,  i.e.,  in  each  instance  F  is  again  zero. 
However,  in  this  case  the  resulting  robust  rules  are  extended 
Bayes.  The  robust  rules  obtained  in  Theorems  4.2,  5.2,  and  5.4 
are  respectively:  Zjv,  Z,  and  Zm,  with  respective  minimax  risks: 
2F„(-V^c),  2Fu(-€),  and  2G„(-e),  where  G„  again  denotes 
the  CDF  of  the  median  based  on  F„. 

These  limiting  cases  provide  useful  upper  and  lower  bounds 
for  the  minimax  risk  for  intermediate  values  of  d. 

6  Robust  Fusion  of  Location  Information 

6.1  Preliminary  Remarks 

In  this  section  we  develop  a  theory  and  methodology  for  robust 
fusion  of  multi- sensor  location  information  based  on  Sections  4 
and  5.  Our  approach  contains  two  distinct  phases: 

•  Phase  I  provides  a  test  of  the  Hypothesis  that  the  location 
data  (2.2)  from  sensor  Si  is  consistent  with  the  location 
data  from  sensor  Sj ,  where  t  <  /,  i.e.,  we  test  the  hypothesis 
that  9i  -  i  <  j. 

•  Phase  II  provides  a  means  of  combining  the  location  data 
from  the  individual  data  sets  which  “pass”  the  Phase  I  test, 
i.e.,  those  deemed  to  be  consistent. 

In  both  phases  of  this  process,  we  seek  procedures  which  are  ro¬ 
bust  to  heavy- tailed  deviations  from  the  nominal  sampling  distri¬ 
bution,  such  as  exhibited  in  e-contamination  uncertainty  classes. 
Our  usage  of  the  terminology  “robust”  is  also  intended  to  imply 
that  the  procedures  have  satisfactory  behavior  when  the  actual 
sampling  distribution  coincides  with  the  nominal,  e.g.,  a  given 
Gaussian  distributipn. 

6.2  Sample  Sizes  and  Uncertainty  Classes 

In  developing  suitable  consistency  tests,  there  are  three  classes 
of  sample  sizes  to  address:  (i)  the  single  sample  case,  Ni  =  1; 
(ii)  the  small  sample  case,  1  <  iVi  <  20;  and  (iii)  the  large 
sample  case,  JV,  >20.  In  defining  these  classes,  it  is  important 
to  observe  that:  (i)  The  transition  (JV  =  20)  between  the  small 
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sample  and  large  sample  cases  is  not  a  precise  threshold  value  — 
the  appropriate  selection  of  this  threshold  is  dependent  on  the 
uncertainty  classes  which  define  the  given  decision  problem;  and 
(ii)  The  sample  sizes  Ni  and  Nj  can  belong  to  different  sample 
size  domains. 

The  selection  of  appropriate  sensor  noise  uncertainty  classes 
{Ti  ;  1  <  *  <  r}  is  an  important  issue  in  the  development  of  a 
methodology  for  robust  fusion  of  multi-sensor  location  informa¬ 
tion.  Since,  at  the  minimum,  we  seek  to  account  for  the  occur¬ 
rence  of  noise  distributions  with  heavy  tails,  it  is  appropriate  to 
consider  6-contamination  uncertainty  classes.  In  the  sequel,  we 
adopt  a  unimodal  symmetric  6- contamination  model  for  each 
sensor  1  <  i  <  r.  In  particular,  we  adopt  the  6i-contaminated 
Gaussian  model  for  sensor  5,-  which  is  defined  by; 

=  (6.1) 

where:  $  =  A/’(0, 1),  the  CDF  H  is  unimodal  and  symmetric 
about  zero,  and  0  <  €,•  <  1/2. 

It  is  also  necessary  to  model  the  a  priori  position  uncertainty 
in  each  sensor.  For  brevity,  we  restrict  our  attention  to  non¬ 
stochastic  models  of  the  form:  \  Wi  \  <  rji,  1  <  *  <  r,  where 
rji  >  0  is  given. 

6.3  Phase  I  —  Robust  Consistency  Tests 

The  following  procedure  provides  a  robust  test  of  the  hypothesis 
that  6i  =  $j,  i  <  j. 

Case  1:  (r/,-  =  0)  Let  -M*,  1  <  *  <  r,  denote  the  class 
of  CDF’s  defined  by  the  centered  sample  median  ZMi  of  Ni  i.i.d. 
samples  with  CDF  F  €  (6.1).  Let  Mij,  1  <  t  <  i  <  r,  denote 

the  class  of  CDF’s  defined  by  the  difference  of  the  centered  sample 
medians  (Zj^i  —  —Oj),  where  the  CDF’s  of  the  centered 

sample  medians  {Zm,  —  ^t)  and  (Zmj  —  $j)  belong,  respectively, 
to  Mi  and  Mj.  It  follows  from  these  definitions  that  the  class 
Mij  is  a  set  of  symmetric  unimodal  distributions.  Further, 

Zm-  ~  ZMj  —  ^i  —  9 3  +  (6-2) 

where:  the  CDF  of  Vij  belongs  to  Mij’,  and  the  a  priori  un¬ 
certainty  in  6i  —  $j  is  given  by  the  interval  [— djj,  dtj],  where 
dij  =  di  -|-  dj. 

Hence,  we  can  construct  a  robust  fixed  size  (2e)  confidence 
procedure  for  Oi  —  6j.  The  parameter  e  is  selected  by  the  decision 
maker  and  denotes  his  tolerance  to  small  errors  between  9i  and 
Oj.  The  desired  procedure  [^*-c,  e]  is  obtained  via  Theorems 
5.2  and  5.4.  Finally,  the  test  of  the  hypothesis  0i  =  $j  is  obtained 
as  follows:  we  reject  9i  =  if  0  ^  [tf*  -  e,  6*  -1-  c].  From  this 
test  we  also  obtain  the  probability  that  9i  —  9j  G  [6*  —  e,  -H  e]. 
Examples  of  applications  of  this  class  of  robust  consistency  tests 
appears  in  McKendall  and  Mintz  (1988), 

Case  2:  (t/,-  >  0)  If  the  uncertainties  ??,•,  1  <  i  <  r,  are 
suitably  small,  then  the  previous  test  can  be  applied  with  only  a 
small  modification  in  the  definition  of  the  uncertainty  class  Mij. 
The  details  appear  in  McKendall  and  Mintz  (1988). 

6.4  Phase  II  —  Robust  Fusion  of  Consistent  Multi- 
Sensor  Location 

Information 

The  following  procedure  provides  a  robust  estimate  of  the  com¬ 
mon  location  parameter  9ofr  sensor  data  sets,  r  >  3.  We  observe 
at  the  outset  that,  when  Vj  and  V2  possess  very  heavy  tails,  it  is 
generally  not  useful  to  attempt  to  combine  only  two  observations 
of  the  form: 

Z2  =  9+V2. 

For  example,  if  Vi  and  V2  are  independent  Cauchy  C(0, 1)  ran¬ 
dom  variables,  then  any  convex  combination  of  Zi  and  Z2  will  be 
a  C(^,  1)  random  variable.  Further,  there  are  random  variables 
with  continuous  unimodal  symmetric  density  functions  whose 
sample  mean,  for  any  sample  size  iV  >  1,  has  greater  variability 
then  any  of  its  N  i.i.d.  components.  (See  for  example.  Brown 


and  Tukey  (1946).) 

Case  1:  (7/,  =  0)  Let  {Zm;  denote  the  sample 

medians  of  r  consistent  data  sets  with  common  location  param¬ 
eter  9.  In  order  to  simplify  the  exposition,  we  further  assume 
that  the  r  sample  medians  are  identically  distributed.  Let 
denote  the  median  of  the  {Zm,  Let  Ma  denote  the 

uncertainty  class  of  the  centered  sample  median  Zma  “  Each 
CDF  F  G  Ma  is  unimodal  and  symmetric  about  zero.  Thus,  we 
can  apply  Theorem  5.2  to  obtain  a  robust  fixed  size  confidence 
procedure  —  e,  e]  for  Examples  of  applications  of 
this  class  of  confidence  procedures  for  the  robust  fusion  of  con¬ 
sistent  multi- sensor  location  information  appears  in  McKendaU 
and  Mintz  (1988). 

Case  2:  (iji  >  0)  If  the  uncertainties  t/,-,  1  <  *  <  r,  are 
suitably  small,  then  the  previous  robust  confidence  procedure  for 
estimating  9  can  be  applied  with  only  a  small  modification  in  the 
definition  of  the  uncertainty  class  Ma^  The  details  appear  in 
McKendall  and  Mintz  (1988). 

6.5  Other  Research  Issues  and  Further  Results 

One  of  the  sufficiency  conditions  in  the  theory  of  robust  fixed  size 
confidence  procedures  delineated  in  Sections  4  and  5  of  this  paper 
is  that  the  value  of  e  >  B  —  a  given  bound.  The  delineation  of 
a  complete  theory  requires  that  we  also  consider  the  case  where 
e  <  B.  It  has  been  shown,  by  Martin  and  Mintz  (1985),  that 
where  c  <  it  is  necessary  to  consider  randomized  decision 
rules  to  obtain  robust  fixed  size  confidence  procedures.  Further 
results  on  the  robust  fusion  of  location  information  pertaining  to 
asymptotic  theory,  a-trimmed  means,  randomized  decision  rules, 
and  nonmonotone  decision  rules  appear  in  McKendall  and  Mintz 
(1988). 
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A  MULTI-FREQUENCY  MULTI-SENSOR  DATA  FUSION  SYSTEM 
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The  Prototype  Information  Correlation  Exploitation  System  (PICES)  is  an  Office  of  Naval 
Technology  6.2  level  exploratory  development  program  for  an  advanced  tracker/correlator  system  and 
is  managed  through  the  Integrated  Ocean  Surveillance  Block  at  the  Naval  Ocean  Systems  Center.  This 
program  addresses  the  needs  of  both  the  Navy's  afloat  and  ashore  ocean  surveillance  nodes  and  the 
needs  of  the  surveillance  community  in  general  to  correlate  and  fuse  multi-frequency,  multi-sensor 
contact  report  data  for  the  detection,  tracking,  and  classification  of  multiple  surface  and  airborne 
targets.  PICES  effectively  addresses  surveillance  operator  overload  due  to  the  heavy  decision 
processes  and  manual  interactions  associated  with  sensor  data  correlation,  sensor  ambiguity 
resolution  and  fusion,  and  target  track  generation,  by  largely  automating  these  surveillance 
functions.  This  paper  described  PICES'  features  and  presents  results  generated  from  data  inputs  at 
both  the  contact  report  and  raw  sensor  output  level. 

PICES  is  a  component  of  a  research,  development,  test  and  evaluation  (RTD&E)  system  using  a 
multiple  hypothesis  tracker/correlator  algorithm.  The  system  is  running  at  NOSC  and  is  implemented 
on  a  VAX  11/785  with  TEKTRONIX  equipment  for  graphical  display.  The  RTD&E ^  system  consists  of 
simulated  and  real  multi-sensor  data  drivers,  fusion  algorithms,  Kalman  filtering  for  target  state 
updating,  various  target  kinematic  models,  a  customized  database  management  system,  measures  of 
performance,  and  a  powerful  user  query  and  real  time  graphical  display  interface. 

The  result  of  PICES  development  is  an  integrated  tracking,  data  correlation,  and  multi¬ 
frequency,  multi-sensor  data  fusion  system  that  automatically  generates  surveillance  scene 
hypotheses  (possibilities  due  to  information  ambiguities)  and  ranks  them  on  the  basis  of  all  sensor 
information  available  to  the  system  at  a  given  time.  The  system  has  demonstrated  a  powerful 
capability  to  automatically  fuse  a  variety  of  contact  report  data  types  resulting  in  great 
alleviation  of  operator  workload.  PICES  handles  the  following  information  types:  unique  attributes, 
geopositional,  range  and  bearing,  and  ELINT  parametrics.  The  query  and  display  capability  allows 
very  effective  operator  interaction  with  the  system.  The  operator  can  generate  graphical  track 
displays,  track  overlays  (for  example,  comparing  ground  truth  with  hypothesis  predictions)  and 
symbology  along  with  hypothesis,  target,  and  sensor  information.  In  addition,  the  operator  can  use 
his  surveillance  experience  to  override  and  correct  system  track  predictions  and  surveillance  data 
inputs,  when  necessary. 

The  paper  discusses  PICES  operation  at  the  sensor  output  level  on  real  HICAMP  infrared 
imagery.  Multi-band  data  was  fused  to  track  weak  airborne  target  signals  in  high  noise  backgrounds, 
where  single  band  tracking  could  not  be  performed.  Finally,  we  discuss  PICES  brassboard  prototype 
microcomputer  implementation  for  potential"  transition  to  fleet  operational  use  and  describe  future 
program  direction. 

(1)  Code  743,  San  Diego,  CA,  92152,  (619)  553-2486 

(2)  9363  Towne  Centre  Drive,  San  Diego,  CA  92121,  (619)  455-5530 
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_ INTRODUCTION 


The  Navy  and  other  DOD  services  are  faced 
with  ever  increasing  surveillance  operational 
requirements.  These  increasing  surveillance 
requirements  are  driven  by  a  number  of  factors.  One 
of  the  most  important  is  the  need  to  maintain  an 
ever-increasing  area  of  coverage,  in  order  to 
counter  the  threat  posed  by  more  powerful  hostile 
missiles  and  aircraft  and  support  over-the-horizon 
targeting  of  our  own  weapon  assets.  Furthermore, 
the  need  for  new  and  improved  sensor  systems  and 
communication  channels  creates  an  information 
explosion  for  the  surveillance  system.  The  general 
requirements  of  increased  timeliness,  accuracy,  and 
throughput  for  surveillance  product  generation 
demands  that  the  sensor  data  correlation  and  fusion 
and  multi-target  track  generation  processes  be 
highly  efficient,  largely  automated  processes. 
Therefore,  a  critical  aspect  of  evolving 
surveillance  systems  is  the  ability  to  fuse  data 
from  different  sensor  systems  in  order  to  generate 
scene  descriptions  which  are  high  in  information 
content  and  low  in  ambiguity. 

The  Prototype  Information  Correlation 
Exploitation  System  (PICES)  is  an  Office  of  Naval 
Technology  6.2  level  exploratory  development  program 
for  an  advanced  tracker/correlator  system  and  is 
managed  through  the  Integrated  Ocean  Surveillance 
Block  at  the  Naval  Ocean  Systems  Center.  This 
program  addresses  both  the  needs  of  the  Navy's 
afloat  and  ashore  ocean  surveillance  nodes  and  the 
needs  of  the  surveillance  community  in  general,  to 
correlate  and  fuse  multi-frequency,  multi-sensor 
report  data  for  the  detection,  tracking,  and 
classification  of  multiple  surface  and  airborne 
targets.  PICES  is  a  multi-hypothesis  computer  based 
tracker/correlator  system  with  a  powerful  operator 
interface.  PICES  effectively  addresses  surveillance 
operator  overload  by  largely  automating  the 
surveillance  functions.  This  paper  describes  PICES 
and  presents  results  generated  from  data  inputs  at 
both  the  contact  report  and  raw  sensor  output  level. 


ZJl _ BENEFITS 

In  this  section  we  describe  some  of  the 
advantages  anticipated  as  a  result  of  replacing 
traditional  manual  tracking  and  correlation  with  an 
automated  multi-hypothesis  tracker/correlator  system 
such  as  PICES. 

ZU _ Relieve  Surveillance  Operator  Overload 

If  an  operator  is  attempting  to  perform  the 
tracking  and  correlation  functions  manually,  he  may 
become  overwhelmed  by  the  sheer  volume  of  reports, 
resulting  in  reports  being  disregarded,  or 
improperly  correlated  with  existing  tracks.  In 
addition,  when  tracking  a  large  number  of  targets, 
an  operator  may  find  it  very  difficult  to  make 
correlation  decisions  due  to  the  number  of  variables 
which  must  be  considered  when  making  decisions,  such 
as  attribute  mismatches,  geopositional  proximity, 
and  ELINT  parametric  correlation. 


Furthermore,  given  highly  ambiguous  data,  an 
operator  may  be  forced  to  delay  making  firm 
correlation  decisions  until  further  data  is 
received,  meaning  he  must  not  only  consider  the  new 
data  arriving,  but  must  also  keep  in  mind  the  impact 
of  the  new  data  on  previous  tentative  correlations. 
The  use  of  a  computer  based  correlation  system  can 
relieve  operator  workload  and  greatly  increase 
throughput  in  processing  large  amounts  of  data. 

ZZ _ Improved  CdtreigtiQ.n.-apd.  Tracking 

In  addition  to  its  ability  to  process  a  large 
quantity  of  data,  PICES  is  capable  of  providing  a 
higher  quality  of  output  than  can  be  provided  by 
manual  processing,  or  by  automatic  processing  with 
less  powerful  algorithms.  Accurate  tracking  is 
achieved  by  the  use  of  a  Kalman  filter.  Improved 
correlation  performance  results  from  the  multi¬ 
hypothesis  approach.  Also,  the  retention  of 
multiple  views  of  a  surveillance  scene  can  provide 
valuable  information  in  a  tactical  situation 
concerning  the  potential  number  and  location  of 
hostile  forces. 

3.0  APPLICATIONS 

PICES  has  applications  in  any  situation  where 
decisions  are  being  made  as  to  the  partitioning  of 
an  incoming  ambiguous  data  set  into  groups  of 
related  measurements.  In  the  following  paragraphs 
we  describe  several  potential  application  areas  for 
PICES. 

3.1  Ocean  Surveillance 

The  original  purpose  for  creating  the  multi- 
hypothesis  system  was  to  apply  the  PICES  technology 
to  sensor  data  correlation  problems  in  ocean 
surveillance.  These  problems  include  wide  area 
ocean  surveillance  and  anti-submarine  warfare. 

2^ _ Targeting 

PICES  can  be  used  in  weapons  targeting  by 
providing  target  localization  information,  in  the 
form  of  a  target  latitude,  longitude,  and  position 
uncertainty  ellipse.  In  addition,  PICES  has  a  built 
in  hit  probability  function  which  calculates  the 
likelihood  of  hitting  a  selected  target  with  a 
weapon.  This  hit  probability  can  be  used  in  a 
tactical  situation  to  aid  in  the  weapon  release 
decision. 

ZZ — Multi-Frecfuencv.  Multi -Sensor  Data  Fusion 

The  current  work  addresses  the  correlation 
and  fusion  of  sensor  contact  reports  (i.e.  ocean 
surveillance  products)  for  target  track  generation. 
These  reports  have  been  generated  by  surveillance 
operators  and  are  therefore  evaluated  sensor 
contacts.  Our  approach  can  also  be  adopted  to  the 
fusing  of  unevaluated  multi-frequency,  multi-sensor 
data.  This  data  for  fusion  can  be  in  a  variety  of 
forms:  multi-spectral  infrared  and  radar  imagery. 


210 


radar  parameters  such  as  range,  doppler,  bearing,  or 
bearings  only  information  such  as  that  received  from 
acoustic,  IRST,  and  ESM  sensors.  Also,  multi-sensor 
attribute  information  can  be  incorporated  in  the 
fusion  process.  The  fusing  of  unevaluated  sensor 
data  poses  an  interesting  tracking  problem, 
particularly  in  the  case  of  dim  target  tracking, 
where  target  detection  takes  place  only  at  the 
expense  of  a  high  number  of  false  alarms.  An 
example  of  using  PICES  for  the  fusion  of  multi- 
spectral  infrared  imagery  data  for  target  tracking 
with  a  moderate  number  of  false  alarms  is  discussed 
in  Section  6.3. 


5.1  Kalman  Filter  Trackex 

PICES  uses  an  extended  Kalman  filter  tracker. 
The  Kalman  filter  state  variables  are  target 
latitude  and  longitude,  along  with  the  corresponding 
velocities.  Either  a  straight  line  or  an  Integrated 
Ornstein  Uhlenbeck  (lOU)  maneuvering  target  motion 
model  may  be  chosen  at  run  time.  Input  reports  may 
contain  position  estimates  in  the  form  of  a 
latitude,  longitude,  and  error  ellipse,  or  in  the 
form  of  a  bearing  wedge  with  minumum  and  maximum 
range.  Both  types  of  reports  may  be  mixed  in  the 
same  scenario. 


:^.4  Sensor  Systems  Evaluation 


5.2  Hypothesis  Management 


As  a  tracking  and  correlation  testbed,  PICES 
can  provide  data  useful  in  the  design  and  evaluation 
of  sensor  systems.  Using  real  or  simulated  input 
data,  the  performance  of  different  sensor  mixtures 
and  sensor  parameters  can  be  measured.  For  example, 
PICES  could  be  used  to  test  the  effectiveness  of 
combining  data  from  a  highly  geopositionally 
accurate  sensor  which  does  not  provide  attributes, 
with  a  sensor  which  is  less  positionally  accurate, 
but  carries  some  attribute  information.  Measures  of 
performance  generated  by  PICES  can  form  the  basis  of 
comparison  between  two  sensor  configurations. 

40  PICES  TESTBED  HARDWARE /SOFTWARE 

Figure  1  shows  a  block  diagram  of  the  current 
PICES  hardware  configuration.  The  central  computer 
responsible  for  performing  the  Kalman  filtering, 
hypothesis  management,  and  database  query  processing 
is  a  DEC  VAX  11/78  5  running  the  VMS  operating 
system.  The  present  PICES  hardware  system  resides 
in  a  NOSC  research  and  development  environment. 
Contact  reports  are  stored  in  a  disk  file  and  are 
read  in  sequentially  and  processed  by  the  PICES 
algorithm.  Any  graphical  plots  which  the  user  has 
requested  are  automatically  updated  after  each 
report  has  been  processed.  In  order  to  minimize  the 
impact  on  processing  speed  due  to  the  updating  of 
the  graphics  display,  a  TEKTRONIX  4132  Unix 
workstation  is  used  to  offload  graphics  processing 
from  the  VAX.  The  VAX  sends  the  minimal  amount  of 
information  needed  to  update  the  display  over  the  RS 
232  interface  to  the  TEKTRONIX  4132.  The  4132  then 
expands  the  command  and  communicates  the  changes  to 
the  412  9  display  terminal  over  a  high  speed  DMA 
interface.  The  use  of  parallel  graphics  processing 
has  allowed  PICES  to  maintain  its  real  time 
graphical  output  with  minimal  impact  on  the  speed  of 
processing  input  reports . 


Given  a  collection  of  target  position 
reports,  there  are  a  large  number  of  ways  in  which 
the  reports  may  be  grouped  together  to  form  tracks. 
Each  possible  grouping,  or  partition,  of  the  input 
reports  into  tracks  is  called  a  hypothesis.  PICES 
is  an  example  of  a  multi-hypothesis  tracker,  which 
means  that  more  than  one  correlation  hypothesis  is 
retained  after  processing  a  contact  report. 


KALMAN  FILTERING 
HYPOTHESIS  MANAGEMENT 
DATABASE  MANAGEMENT 
GRAPHICS  UPDATES 


GRAPHICS  PROCESSING 
MESSAGE  PASSING 


GRAPHICS  DISPLAY  (CRT) 
LOCAL  PAN  AND  ZOOM 
DIALOG  AREA 


DATABASE  QUERIES 
REPORT  GENERATION 
TARGETING 


PICES  HARDWARE  BLOCK  DIAGRAM 
FIGURE  1 


Further  processing  power  is  available  on  the 
4129  graphics  terminal,  which  allows  the  operator  to 
pan  and  zoom  the  display  region  without  intervention 
from  the  4132  or  the  VAX.  The  4129  also  has  a 
dialog  area  where  status  information  and  database 
query  responses  are  displayed. 

Figure  2  shows  a  block  diagram  of  the  major 
software  components  of  the  PICES  testbed.  At  the 
heart  of  the  system  is  the  data  fusion  algorithm, 
which  is  being  fed  from  data  generated  by  the 
scenario  generator  (or  other  data  source) .  The  user 
interacts  with  PICES  through  the  query  language  and 
graphics  interface.  Scenarios  can  be  constructed 
based  on  an  analysis  of  available  ocean  surveillance 
products  (OSP)  .  The  KBES  block  refers  to  a 
knowledge  based  expert  system  decision  aid  which  is 
not  yet  part  of  PICES,  but  may  be  incorporated  in 
the  future. 


5.0  SYSTEM  nVERVTF.W 

In  this  section'  we  describe  important  PICES 
features  and  testbed  components  shown  in  Figure  2. 


PICES  -  MULTI-HYPOTHESIS  DATA  FUSION  SYSTEM  TESTBED 


FIGURE  2 
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Figure  3  illustrates  the  formation  of  the 
hypothesis  "tree".  Reports  hypothesized  to  be  from 
a  single  target  are  separated  by  commas,  while  those 
reports  hypothesized  to  be  from  different  targets 
are  separated  by  vertical  lines.  The  base  of  the 
tree  consists  of  the  single  report  "a".  When  report 
"b"  is  received,  two  hypotheses  can  be  formed.  The 
upper  branch  represents  the  hypothesis  that  reports 
"a"  and  "b"  are  from  the  same  target,  while  the 
lower  branch  represents  the  hypothesis  that  the 
reports  are  from  two  distinct  targets.  The  receipt 
of  report  "c"  results  in  a  total  of  5  possible 
hypotheses.  It  is  easily  seen  that  if  all  possible 
hypotheses  are  retained,  the  hypothesis  tree  will 
rapidly  grow  to  an  unmanageable  size  as  more  reports 
are  processed.  Thus,  low  scoring  hypotheses  are 
pruned  so  that  the  number  of  hypotheses  retained 
after  processing  a  contact  report  remains  roughly 
constant . 

The  advantage  of  maintaining  multiple 
hypotheses  is  that  decisions  concerning  ambiguous 
reports  may  be  delayed  until  further  data  is 
available.  An  example  where  this  approach  is 
effective  is  during  track  initiation.  When  the 
first  report  on  a  target  has  been  received,  it  is 
generally  impossible  to  estimate  its  velocity. 
Typically,  a  velocity  of  0  is  assigned  to  the  track. 
When  another  report  arrives  near  the  previous 
report,  it  may  not  yet  be  clear  whether  this  report 
is  from  the  same  target  as  seen  before,  or  is  from  a 
new  target.  A  multi-hypothesis  tracker  would 
therefore  form  two  hypotheses  concerning  the  origin 
of  the  second  report.  The  hypothesis  that  the 
second  report  is  from  the  same  target  as  the  first 
would  allow  the  velocity  of  the  target  to  be 
estimated.  Now,  if  a  third  report  comes  in,  its 
position  can  be  compared  to  the  position  expected 
from  a  target  moving  with  the  velocity  calculated 
from  the  previous  hypothesis.  If  the  positions 
match  closely,  we  now  have  evidence  that  the 
previous  hypothesis  linking  the  first  two  reports  is 
correct.  The  scoring  algorithm  would  then  give  the 
single  target  hypothesis  a  higher  score  than  the 
JiypcJ'thesis  that  all  reports  are  from  different 
targets.  As  more  reports  are  received  from  the 
target,  the  single  target  hypothesis  would  continue 
to  grow  in  probability. 

When  an  input  report  is  received,  a  number  of 
correlation  hypotheses  are  generated  and  scored. 
Prior  to  formation  of  hypotheses,  the  incoming 
report  is  compared  with  each  of  the  existing  tracks 
in  the  database  for  geopositional,  attribute,  and 
FLINT  parametric  feasibility.  This  step,  known  as 
gating,  eliminates  candidate  tracks  which  have 
virtually  no  probability  of  associating  with  the 
incoming  report  and  helps  keep  the  number  of 
hypotheses  which  must  be  scored  and  maintained  to  a 
manageable  level. 

MULTI-HYPOTHESIS  DATA  FUSION 


Assume  three  unequated  contact  reports.  Five  hypotheses  ranging  from 
1  target  to  3  targets  are  generated.  False  alarms  are  ignored  in  this  example. 

FIGURES 


Following  the  gating  step,  a  score  is 
calculated  for  each  of  the  possible  associations  of 
the  incoming  report  with  existing  tracks,  along  with 
the  probability  that  the  report  is  from  a  new 
target.  If  the  incoming  report  contains  FLINT 
parameter  data  as  well  as  geopositional  data,  the 
association  score  reflects  both  types  of  data. 


A  number  of  operator  interface  features  have 
been  incorporated  into  PICES,  to  make  it  more 
productive  as  a  research  testbed,  and  to  ultimately 
allow  it  to  make  the  transition  to  an  operational 
setting.  The  following  paragraphs  describe  some  of 
the  operator  interface  features  of  PICES, 

Database  Management  and  Query  System 

One  of  the  most  powerful  and  useful  aspects 
of  the  PICES  user  interface  is  the  database  query 
language.  Using  a  syntax  similar  to  the  commands 
used  to  operate  commercial  relational  database 
systems,  the  operator  is  able  to  call  up  for  tabular 
display  any  information  stored  in  the  PICES 
database.  The  information  most  often  accessed  is 
the  hypothesis  list  and  the  list  of  tracks  contained 
in  the  top  hypothesis.  The  user  may  specify  which 
fields  in  the  database  record  he  wishes  to  see,  and 
may  place  conditions  on  the  fields,  or  sort  the 
displayed  records  according  to  a  field.  For 
example,  to  see  a  list  of  the  tracks  (referred  to  in 
PICES  as  "clusters")  contained  in  the  best  (most 
probable)  hypothesis  which  are  known  to  be  flying  a 
Soviet  flag,  the  user  may  issue  a  display  command  as 
follows : 

DISPLAY  CLUSTERS  FROM  HYPOTHESIS  MAXIMUM  FOR  FLAG  EQ 
SOVIET 

As  shown  in  Figure  4,  PICES  responds  with  a 
list  of  all  the  items  from  the  database  which 
satisfy  the  conditions.  To  save  on  keystrokes,  the 
above  command  could  be  shortened  by  using  three  or 
four  character  abbreviations. 

In  addition  to  the  ability  to  query  the 
database,  the  user  is  also  provided  with  an  editing 
capability.  For  example,  if  an  operator  has 
knowledge  of  a  ship's  identity  which  is  not 
available  to  PICES,  he  may  edit  the  name  field  in 
the  ship's  database  record  to  fill  in  the  missing 
information.  The  operator  may  also  delete 
hypotheses  or  tracks  if  he  believes  PICES  is  in 
error.  We  felt  that  the  editing  capability  was 
important  so  PICES  can  work  with  the  human  operator, 
who,  although  slow  compared  to  the  speed  of  a 
computer,  may  have  knowledge  and  experience  which 
the  tracker  software  does  not  possess. 


-xJisp  dus  from  hyp  max  (or  (lag  aq  sv 

INDEX  TIME  LAT  LONG  SCORE  CONTACT  NAME  CLASS  TYPE  FL  NOS 

3  021536:DEC:86  35.46  19.81  1.00  NAV  GROZNY  KYNDA  CG  SV  200 

14  021536:DEC:B6  35.39  20.93  1.00  NAV  LENINGRAD  KIEV  CV  SV  200 


FIGURE  4 
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Real  Time  Graphics  Display 

The  most  important  user  interface  component 
present  in  PICES  is  the  graphical  display.  The  user 
may  request  plots  of  the  tracks  being  formed  using 
the  full  power  of  the  query  language  to  place 
conditions  on  the  displayed  data.  As  reports  are 
processed,  the  tracks  on  the  display  are 
automatically  updated  to  reflect  the  new  data.  For 
example,  the  operator  may  wish  to  restrict  his 
attention  to  a  particular  geographic  region,  and  may 
be  interested  only  in  naval  vessels.  By  issuing  a 
conjmand  to  PICES,  the  conditions  would  be  evaluated 
for  each  track  in  the  scenario,  and  only  those 
tracks  satisfying  the  conditions  would  be  displayed. 

This  graphics  capability,  which  would  be 
required  in  an  operational  setting,  has  also  proved 
invaluable  during  the  research  and  development  phase 
of  the  project.  Changes  to  the  algorithm  which 
affect  performance  can  be  instantly  seen  as  a  result 
of  the  visual  feedback  provided  by  the  graphics 
display.  For  example,  the  ground  truth  tracks  may 
be  overlaid  with  the  tracks  calculated  by  PICES  for 
coir^jarison.  The  data  which  may  be  plotted  include 
the  raw  sensor  measurements,  the  ground  truth  tracks 
generated  by  the  simulator,  and  the  tracks 
corresponding  to  the  hypotheses  formed  by  the  PICES 
algorithm. 

Figures  4  and  5  illustrate  some  of  the 
graphics  capabilities  of  PICES.  Figure  4  shows  a 
view  of  a  scenario  where  five  vessels  moving  south 
are  entering  a  shipping  lane.  The  user  may  request 
plots  of  targets  of  interest,  as  in  Figure  5  where 
the  user  has  asked  for  a  display  of  only  the  targets 
known  to  be  Soviet. 


Targeting  Figure  of  Merit 


An  interesting  feature  which  was  added  to 
PICES  was  the  calculation  of  a  hit  probability.  The 
operator  may  select  a  target  and  a  search  ellipse, 
and  have  PICES  calculate  the  probability  of  hitting 
the  target  with  a  missile  capable  of  searching  the 
region  inside  the  ellipse  centered  at  the  estimated 
location  of  the  target  of  interest.  This  hit 
probability  takes  into  account  the  possibility  that 
the  target  of  interest  may  not  be  in  the  search 
region,  as  well  as  the  possibility  that  another 
target  may  be  inside  the  region  and  may  be  hit 
instead  of  the  intended  target.  in  a  threat 
situation  at  sea,  this  capability  could  provide  a 
commander  with  valuable  information  which  would  help 
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him  decide  whether  a  weapon  should  be  released  or 
not. 

^■4  Pf^T-fnrmance  Measures  and  Input  Data  Ambiffuit-Y 

A  number  of  performance  measures  have  been 
incorporated  into  PICES  to  aid  in  quantifying  the 
tracking  and  correlating  capabilities  of  the 
algorithm,  as  well  as  evaluating  the  effectiveness 
of  different  sensor  characteristics.  The 

performance  measures  may  be  called  up  at  any  time 
during  the  processing  of  a  simulated  data  set. 
Included  in  the  performance  measures  are  several 
quantities  which  measure  the  ambiguity  present  in 
the  input  data  set.  The  input  ambiguity  is 
important  to  know,  since  performance  of  the 
algorithm  is  always  relative  to  the  difficulty  of 
the  scenario  being  tested.  Work  on  quantifying  the 
ambiguity  of  the  input  data  set  is  continuing.  At 
this  time,  the  following  have  been  implemented: 

Hypothesis  Ratio 

The  hypothesis  ratio  is  one  of  several 
measures  of  the  ambiguity  of  the  input  data  set.  It 
is  calculated  as  the  ratio  of  the  number  of 
hypotheses  which  could  be  formed  after  the  gating 
step  to  the  number  of  hypotheses  which  would  have 
been  formed  in  the  absence  of  gating.  Unambiguous 
data,  for  example  data  in  which  the  targets  are 
widely  separated  geographically  or  have  unique 
attributes,  will  show  a  low  hypothesis  ratio,  as 
most  of  the  potential  associations  will  be  prevented 
during  the  gating  phase  of  the  algorithm.  On  the 
other  hand,  for  highly  ambiguous  data,  gating  will 
provide  a  less  dramatic  reduction  in  the  number  of 
candidate  tracks,  resulting  in  a  large  hypothesis 
ratio. 

Hypothesis  Evolution 

Following  the  processing  of  a  scenario,  a 
plot  of  the  ancestry  of  any  selected  hypothesis  may 
be  obtained.  For  example,  the  best  hypotheses 
present  at  the  end  of  the  scenario  may  be  selected 
for  analysis,  resulting  in  a  plot  of  the  ranking  of 
all  of  its  ancestor  hypotheses  in  the  hypothesis 
tree.  This  is  another  measure  of  the  ambiguity  of 
the  input  data,  since  highly  ambiguous  data  should 
result  in  considerable  fluctuation  in  the  ranking  of 
the  hypotheses,  while  for  unambiguous  data  the  top 
ranking  hypothesis  would  generally  yield  a 
descendant  which  is  also  the  top  ranking  hypothesis. 

Entropy 

As  a  measure  of  correlational  performance, 
one  may  consider  the  track  entropy,  defined  by  the 
following  siammation: 

-LE(A(i, j) /N) *log(A(i, j) /NGT ( j) ) 


where 

A(i,j)  *  number  of  reports  from  ground  truth  target 
j  which  were  assigned  by  the  tracker  to  constructed 
track  i, 

N  “  total  number  of  reports  processed,  and 

NGT(j)  -  niamber  of  reports  from  ground  truth  track  j. 


The  entropy  gives  a  measure  of  the  dispersion 
of  ground  truth  tracks  to  tracks  constructed  by 
PICES.  An  entropy  of  0.0  implies  that  each  ground 
truth  target  has  had  all  of  its  reports  correctly 
assigned  to  a  single  constructed  track.  Large 
entropies  reflect  essentially  a  random  assignment  of 
ground  truth  reports  to  constructed  tracks, 
indicating  poor  correlation  performance.  Entropy 
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based  calculations  applied  to  the  input  data  set  are 
currently  being  investigated  as  a  measure  of  the 
inherent  ambiguity  of  the  scenario. 

5.5  SimulatQX 

In  order  to  provide  controlled  scenarios  for 
testing  the  tracking  and  correlating  algorithms  in 
PICES,  a  scenario  simulator  was  developed. 
Scenarios  with  multiple  targets  and  multiple  sensor 
types  may  be  created.  For  example,  sensors  may  be 
modelled  as  active  or  passive,  with  the  passive 
sensors  only  able  to  detect  emitting  targets.  FLINT 
sensors,  which  detect  both  geoposition  and  ELINT 
parameters  such  as  pulse  repetition  interval  (PRI) 
and  scan  rate  (SCAN)  may  also  be  modelled.  Up  to  5 
emitters  may  be  placed  on  each  target,  and  changes 
in  an  emitter’s  PRI  may  be  programmed  to  occur  at 
any  time  during  the  course  of  the  scenario.  By 
turning  all  emitters  on  a  target  off,  emissions 
control  (EMCON)  may  be  simulated. 


variety  of  sensors  in  order  to  investigate  the 
feasibility  of  tracking  targets  in  a  high  density 
shipping  lane.  One  space  based  sensor,  which  gave 
position  information  without  attributes,  was 
included  with  two  land  based  sensors,  which  reported 
some  attribute  information  along  with  position.  The 
Soviet  vessels  went  into  EMCON  as  they  entered  the 
shipping  lane.  A  nxamber  of  scenarios  were  generated 
with  different  values  of  reporting  rates, 
measurement  errors,  and  availability  of  attributes. 
As  a  result  of  these  studies  conclusions  were  drawn 
regarding  the  effectiveness  of  different  sensor 
characteristics  and  sensor  combinations. 
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Sensors  may  be  selected  to  produce 
latitude /longitude  or  bearings  only  type  of  reports, 
and  can  be  assigned  different  probabilities  of. 
detection  and  different  reporting  rates.  Reports 
include  error  ellipse  information  and  may  be  offset 
from  the  ground  truth  location.  Target  attributes 
may  be  given  individual  probabilities  of  detection, 
making  it  possible  to  simulate  sensors  with  any  mix 
of  attribute  data.  Construction  of  realistic 
scenarios  has  been  aided  by  analysis  performed  on 
real-world  data,  revealing  typical  data  rates, 
measurement  uncertainties,  and  availability  and 
types  of  attribute  information  on  incoming  reports. 
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=>  BEST  HYPOTHESIS  AFTER  17  PRIMITIVES 
FIGURE  6 
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E.Q  EXPERIMENTAL  RESULTS 


In  this  section,  we  describe  some 
experimental  results  obtained  by  running  PICES  on 
several  types  of  scenarios.  In  addition,  we 
illustrate  many  of  the  operator  interface  features 
described  earlier. 


surface  2: 
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6.1  Soviet  Attack  Force 


A  scenario  was  created  which  had  5  Soviet 
warships  entering  a  busy  shipping  lane  in  the 
Mediterranean,  travelling  in  the  shipping  lane  for 
two  days,  and  emerging  in  an  attack  force.  Figures 
6  through  8  show  the  five  Soviets  entering  the 
shipping  lane  at  the  start  of  the  scenario,  and  are 
good  examples  of  the  multi-hypothesis  approach  at 
work.  The  Soviets  enter  the  scene  from  the  north, 
and  travel  in  parallel  formation  until  they  reach 
the  shipping  lane.  At  that  time,  they  turn 
southeast  into  the  shipping  lane.  Figure  6  shows 
that  PICES  has  incorrectly  failed  to  link  the 
reports  from  two  of  the  vessels  in  the  best 
hypothesis.  However,  as  shown  in  Figure  7,  the 
correct  linkage  for  the  second  track  from  the  right 
is  retained  in  the  second  best  hypothesis.  The 
track  in  this  hypothesis  will  carry  a  non-zero 
velocity,  making  it  possible  to  accurately  predict 
the  location  of  the  third  report  from  the  target. 
After  more  reports  have  been  received,  the  correct 
hypothesis  for  the  second  track  from  the  right  is 
brought  up  to  the  top  by  the  scoring  algorithm,  as 
shown  in  Figure  8.  This  hypothesis,  since  it  did 
such  a  good  job  of  predicting  the  third  report  from 
the  vessel,  is  rewarded  with  a  large  probability, 
bringing  it  up  from  the  second  most  likely 
hypothesis  to  the  best  hypothesis.  PICES  has  also 
connected  the  reports  from  the  second'  vessel  from 
the  left.  Thus  PICES  has,  in  effect,  corrected  its 
earlier  mistakes,  corrections  which  would  have  been 
difficult  if  multiple  hypotheses  were  not  retained. 

This  scenario,  in  addition  to  being  used  to 
illustrate  the  value  of  a  multi -hypothesis  approach, 
was  also  used  to  test  the  effectiveness  of  mixing  a 
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=>BEST  HYPOTHESIS  AFTER  32  PRIMITIVES 
FIGURE  8 


6.2  ELINT  Scenarios 

A  number  of  scenarios  were  run  to  test  the 
value  of  incorporating  ELINT  scoring  into  the 
algorithm.  The  scenarios  constructed  were  based  on 
the  scenarios  used  to  evaluate  the  POST  (Prototype 
Ocean  Surveillance  Terminal)  and  TEC  (Tactical  Elint 
Correlator)  ELINT  correlators  [Ref.  1]  in  an  OTH-T 
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context.  These  are  prototype  research  and 
development  systems.  In  this  reference,  it  was 
suggested  that  POST  and  TEC  could  be  improved  by 
incorporating  geopositional  scoring  into  the 
correlation  decisions.  Since  PICES  can  use  both 
geoposition  and  ELINT  parameters  in  the  hypothesis 
scoring  process,  it  was  a  good  candidate  for  testing 
that  suggestion. 

Two-Target  Scenarios 

Several  two  target  scenarios,  similar  to  the 
ones  used  in  the  POST/TEC  evaluation,  were  tested 
using  geoposition  only,  and  using  both  geoposition 
and  ELINT  scoring.  In  general,  we  found  that  the 
performance  of  PICES  on  the  two-target  ^  ELINT 
scenarios  using  geopositional  data  only  in  the 
scoring  algorithm  was  at  least  as  good  or  better 
than  the  performance  of  either  POST  or  TEC,  while 
the  performance  of  PICES  with  both  geopositional  and 
ELINT  parameter  scoring  seemed  to  be  significantly 
better.  Since  it  was  not  possible  to  run  PICES  on 
exactly  the  same  data  set  as  was  used  to  test  POST 
and  TEC,  it  is  somewhat  unfair  to  come  to  hard 
conclusions  about  the  performance  of  PICES  relative 
to  the  other  trackers.  However,  it  can  be  concluded 
that  PICES  performed  quite  well  on  these  tests. 

Figure  9  shows  a  close  up  of  a  recoiling 
target  scenario,  illustrating  the  close  proximity  of 
the  targets  at  their  closest  point.  PICES  ran  the 
entire  scenario  without  making  a  single  correlation 
mistake,  using  both  geoposition  and  ELINT  scoring. 
The  ELINT  parameters  of  the  two  targets  were  set 
about  two  standard  deviations  apart.  Without  using 
ELINT  scoring,  PICES  would  have  tended  to  show  the 
tracks  crossing  rather  than  recoiling,  due  to  the 
straight  line  prediction  of  the  Kalman  filter. 
Thus,  in  the  case  of  this  scenario,  the  use  of  ELINT 
scores  made  a  dramatic  improvement  in  the 
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ECAP  Scenario 

In  order  to  test  PICES  on  a  large,  realistic 
ELINT  scenario  a  copy  of  the  ECAP  II  scenario  was 
used  as  input  to  PICES.  The  ECAP  (ELINT  Correlation 
Analysis  Project)  scenario  is  a  large  scenario 
containing  six  Soviet  vessels  and  about  117  other 
vessels  in  the  Straight  of  Gibralter.  ^  The  six 
Soviets  engage  in  a  search  and  rescue  mission,  with 
three  ships  arriving  from  the  Atlantic  and  three 
from  the  Mediterranean.  The  contact  reports  contain 
both  geopositional  and  realistic  ELINT  parametric 
data. 


PICES  was  run  on  the  ECAP  scenario  two  times, 
first  with  ELINT  scoring  disabled,  and  then  with 
ELINT  scoring  enabled.  After  100  contact  reports 
had  been  received,  the  scenario  was  paused  and  the 
measures  of  performance  were  calculated.  The 
entropy  was  found  to  be  more  than  twice  as  large  for 
the  case  where  ELINT  scoring  was  not  used  than  for 
the  ELINT  scoring  case.  Since  large  entropy 
reflects  greater  "confusion",  we  concluded  that  the 
use  of  ELINT  scoring  greatly  aided  the  algorithm  in 
making  the  correct  correlation  decisions. 


Figures  10,  11,  and  12  further  illustrate  the 
benefits  of  ELINT  scoring.  In  the  following 
figures,  values  for  PRI  and  SCAN  have  been  altered 
so  as  not  to  reveal  classified  information.  Figure 
10  is  a  close  up  of  the  raw  measurement  data  from 
three  Soviets  travelling  east,  showing  the  ground 
truth  linkage  of  their  contact  reports.  The  tracks 
show  a  great  deal  of  positional  ambiguity  in  the 
measurement  data,  with  the  uppermost  track  crossing 
over  the  other  two  tracks.  The  table  in  the  figure 
illustrates  the  output  of  a  query  to  the  database 
manager  concerning  the  reports  contained  in  the 
uppermost  track.  Note  that  the  reported  scan  of  410 
for  one  of  the  reports  differs  significantly  from 


INDEX 

1 

13 

20 

37 

67 

80 


TIME 

010025:NOV:83 

010117:NOV:83 

010218:NOV:83 

010304:NOV:83 

011414:NOV:83 

011449:NOV;83 


LAT 

34.40 

34.53 

34.48 

34.92 

35.12 

35.23 


LONG 

-11.60 

-11.05 

-10.23 

-9.93 

-9.17 

-8.82 


TID 

5 

5 

5 

5 

5 

5 


PULSEREP 

1094 

1094 

1096 

1096 

1097 
1092 


SCAN 

400 

399 

397 

398 
410 
396 


=>SOME  ECAP  TRUTH  TRACKS  AND  TABLE  OF  REPORTS  FROM  TRUTH  TRACK  1 


INDEX 

TIME 

LAT 

2 

010029:NOV:83 

34.10 

9 

010116;NOV:83 

34.28 

12 

010117;NOV:83 

34.40 

20 

010218:NOV:83 

34.48 

35 

010229:NOV:83 

34.68 

37 

010304:NOV:83 

34.92 

67 

010414:NOV;83 

35.12 

80 

010449:NOV:83 

35.23 

LONG 

TID 

PULSEREP 

SCAN 

-11.55 

3 

1161 

434 

-11.05 

3 

1162 

434 

-11.18 

4 

1142 

436 

-10.23 

5 

1096 

397 

-10.00 

3 

1164 

431 

-9.93 

5 

1096 

398 

-9.17 

5 

1097 

410 

-8.82 

5 

1092 

396 

=>ELINT  SCORING  DISABLED  -  CONTACT  REPORTS  FROM  TRACK  12 


INDEX 

TIME 

LAT 

LONG 

TID 

PULSEREP 

SCAN 

1 

010025;NOV:83 

34.40 

-11.60 

5 

1094 

400 

13 

010117:NOV:83 

34.53 

-11.05 

5 

1094 

399 

20 

010218-.NOV:83 

34.48 

-10.23 

5 

1096 

397 

37 

010304:NOV:83 

34.92 

-9.93 

5 

1096 

398 

80 

010449:NOV:83 

35.23 

-8.82 

5 

1092 

396 

=>ELINT  SCORING  ENABLED  -  REPORTS  FROM  TRACK  12 


FIGURE  12 


215 


Figure  11  shows  the  tracks  formed  by  PICES 
when ‘ tracking  was  based  on  geoposition  only,  with 
ELINT  tracking  disabled.  Many  correlation  mistakes 
have  been  made,  as  can  be  seen  by  the  "TID",  or 
ground  truth  target  id,  column  in  the  table  for  the 
uppermost  track.  Three  different  targets  have  been 
linked  together  here  as  a  result  of  the  large 
geopositional  ambiguity. 

Finally,  Figure  12  shows  PICES  tracking  the 
three  Soviets  using  ELINT  and  geopositional  scoring. 
The  three  targets  are  correlated  perfectly  with  the 
exception  of  the  report  containing  the  outlying  scan 
value,  which  is  not  linked  with  the  other  reports. 
This  report,  with  track  id  54  in  the  figure,  will 
eventually  be  deleted,  as  it  will  fail  to  be  updated 
by  subsequent  reports. 
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6.3  HICAMP  Infrared  Multi-Spectral  Imagery Fusion 

An  experiment  was  conducted  using  HICAMP 
infrared  imagery  to  see  if  the  PICES  algorithm  could 
be  used  for  low  signature  target  detection.  This 
situation  differs  significantly  from  the  ocean 
surveillance  scenarios  described  previously  due  to 
the  extremely  high  data  rate,  and  the  presence  of  a 
large  number  of  false  alarms.  A  data  set  containing 
simultaneously  measured  short  and  long  wave  infrared 
imagery  was  obtained.  Two  targets  were  present,  an 
F14  flying  parallel  to  a  drone.  To  present  a  more 
challenging  problem  to  PICES,  the  drone,  which  was 
much  dimmer  than  the  F14,  was  chosen  for  tracking. 

The  IR  data  consists  of  about  100  frames  of 
pixel  amplitudes,  corresponding  to  the  energy 
observed  in  the  sensor  element  during  the  duration 
of  the  frame.  Several  preprocessing  steps  were 
necessary  before  the  data  could  be  input  to  PICES. 
First,  successive  frames  were  differenced  to  remove 
stationary  background  values.  Next,  a  peak 
detection  step  was  applied  to  the  data  to  be  sure 
that  any  target  detections  were  confined  to  a  single 
pixel.  Finally,  to  reduce  noise,  the  data  was 
thresholded  by  retaining  only  those  pixels  which 
exceeded  a  threshold.  Any  pixel  which  remained  with 
a  non  zero  value  after  the  thresholding  was 
converted  to  a  position  report  and  input  to  PICES. 
A  typical  frame  of  data  consisted  of  about  10  noise 
values  (false  alarms) .  A  target  detection  would 
occur  about  every  4  or  5  frames. 

Figure  13  shows  the  candidate  tracks  being 
formed  by  PICES  just  after  the  drone  has  entered  the 
scene  in  the  long  wave  data  set.  The  tracks  formed 
by  noise  points  tend  to  be  short,  and  can  usually  be 
purged  as  a  result  of  not  being  updated  often 
enough,  or  because  of  velocities  which  would  not  be 
characteristic  of  a  target  of  interest . 

Figure  14  shows  the  beginning  of  a  track  for 
the  drone  showing  up  in  the  lower  right  quadrant  of 
the  figure  after  the  target  has  been  in  the  picture 
for  several  frames.  Finally,  Figure  15  shows  a 
clear  track  being  formed  for  the  drone.  Notice  in 
comparing  the  three  figures  that  noise  tracks  tend 
to  come  and  go,  and  are  held  to  a  relatively 
constant,  and  manageable,  level.  In  practice,  some 
decision  criteria  will  be  employed  to  determine  if  a 
track  represents  a  detection.  The  table  in  Figure 
15  contains  columns  for  velocity  {LA_VEL  and 
LO_VEL) ,  average  report  to  track  association 
probability  (ASSOC_R)  ,  and  total  number  of  reports 
in  a  track  (TOTALH) .  These  quantities  can  be  used 
by  PICES  to  make  deletion  decisions  concerning  false 
tracks .  The  drone  in  the  above  data  set,  while  not 
entirely  invisible  to  the  trained  eye  in  the  raw 
data,  was  quite  difficult  to  see,  so  that  the 
ability  of  PICES  to  detect  and  track  it  was 
significant . 
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TRACKING  INFRARED  DATA 
FIGURE  13 


O 

TARGET  APPEARS  AS  CLUSTER  15 
FIGURE  14 


0^® 

[] 

[] 

[] 


¥ 


[] 

[] 


ffl 


□ 


=>disp  clus  from  hyp  max  for  totalh  get  2  sort  totalh 


INDEX 

LAT 

LONG 

LA  VEL 

LO  VEL 

ASSOC  R  TOTALH 

19 

23.97 

22.26 

0.00 

7.30 

0.801  3 

8 

29.36 

22.34 

4.10 

-2.82 

0.794  5 

18 

27.66 

26.53 

7.68 

0.00 

0.949  13 

FIGURE  15 

In  addition  to  the  work  done  in  tracking  the 
drone  in  the  long  wave  data,  some  experimentation 
was  done  on  fusion  of  the  short  and  long  wavelength 
data  sets.  Unfortunately,  upon  close  examination  of 
the  short  wave  data  set,  it  appeared  that  the  drone 
was  not  present,  while  the  F4  was  easily  visible  in 
both  data  sets.  Therefore,  to  experiment  with 
multiband  fusion,  we  artificially  dimmed  the  F4  in 
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the  two  data  sets  by  randomly  deleting  detections 
until  the  target  could  not  be  seen  or  tracked  in 
either  data  set  alone.  The  two  data  sets  were  then 
merged  and  input  to  PICES,  which  was  able  to  gather 
enough  information  in  the  fused  data  set  to  track 
the  target.  This  example  resulted  in  the  fusion  of 
individual  band  target  signal  to  noise  ratios  of 
approximately  2  dB  to  yield  an  enhanced  target  track 
signal  to  noise  ratio  of  5  dB.  The  algorithm  has 
not  been  fine  tuned  for  this  multi -freguency  imagery 
problem,  so  that  further  work  is  needed  to  quantify 
the  limitations  of  the  algorithm  for  higher  false 
alarm  rates  and  lower  target  signal. 


7.0  CURRENT  WORK 

In  this  section,  we  briefly  mention  the 
current  work  on  the  PICES  project.  One  of  the  major 
thrusts  of  the  current  effort  is  to  transition  PICES 
from  a  research  and  development  environment  into  an 
operational  setting. 

Microcomputer  Implementation 

An  Intel  80386  based  microcomputer  has  been 
purchased  as  a  platform  for  the  PICES  algorithm. 
The  computer  is  supplied  with  a  high  performance 
graphics  board,  a  19  inch  color  monitor,  and  a 
numeric  coprocessor.  We  anticipate  performance 
con?>arable  to  the  performance  we  have  obtained  with 
the  combination  of  the  VAX,  TEKTRONIX  4132 
workstation,  and  TEKTRONIX  4129  graphics  terminal, 
at  a  fraction  of  the  cost.  The  move  to  put  PICES  on 
a  microcomputer  is  aimed  at  making  it  a  low  cost, 
portable  system  for  use  in  an  operational 
environment. 

Live  Sensor  Report  Data  Interface 

A  link  to  real  time  data  will  be  provided  for 
PICES  by  connecting  PICES  to  the  an  ocean 
surveillance  product  report  communications 
interface.  This  will  move  us  further  toward  our 
goal  of  an  operational  home,  and  will  be  a  source  of 
real  data  for  analysis  and  refinement  of  the 
tracking  and  hypothesis  generation  algorithms. 

Live  Sensor  Report  Data  Analysis 

A  large  amount  of  data  gathered  from  the 
communications  link  will  be  analyzed  for  such 
characteristics  as  report  rate,  report  accuracy  and 
report  content.  The  data  gathered  from  this  study 


will  help  us  create  more  realistic  scenarios,  and 
may  suggest  improvements  to  the  algorithm. 


ft , Q  FUTURE  WORK 

A  major  goal  of  future  work  on  PICES  will  be 
to  transition  the  system  to  a  Navy  operational 
setting.  Some  of  the  issues  which  need  to  be 
addressed  include  the  possible  need  for  higher 
confutation  speeds,  an  improved  user  interface,  and 
fine  tuning  of  the  tracking  and  scoring  methods.  In 
addition,  steps  must  be  taken  to  ensure  that  PICES 
is  compatible  with  existing  data  communications 
interfaces.  As  new  high  performance  computer 
hardware,  such  as  transputer  boards  or  array 
processors,  becomes  available  its  possible  use  in 
the  tracker/correlator  system  will  be  studied. 

An  additional  application  area  for  the  PICES 
technology  is  in  the  area  of  data  fusion.  Further 
work  is  needed  to  identify  algorithmic  and  hardware 
limitations  in  addressing  low  signal  to  noise  target 
tracking  and  the  associated  high  false  alarm  rate. 
The  application  of  the  multi -hypothesis  approach  to 
the  tracking  of  targets  characterized  by  low  imagery 
contrast  and  low  radar  cross  section  (RCS)  using  a 
variety  of  sensor  types  would  be  an  interesting 
multi-frequency,  multi-sensor  fusion  problem. 

Q ■ 0  rONCLUSIQNS 

The  work  done  to  date  on  PICES  has  resulted 
in  a  tracker/correlator  system  which  includes  many 
of  the  recent  advances  in  tracker/correlator 
algorithms,  as  well  as  a  powerful  user  interface. 
The  effectiveness  of  the  multi-hypothesis  approach 
has  been  demonstrated  in  a  research  environment. 
Future  work,  including  processing  of  live  ocean 
surveillance  sensor  report  data,  will  allow  us  to 
evaluate  the  performance  of  the  microcomputer 
implementation  of  PICES  on  realistic  scenarios.  The 
next  challenge  is  to  gain  operational  experience, 
and  use  the  knowledge  gained  to  suggest  improvements 
to  the  algorithms  and  user  interface. 
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ABSTRACT 


Although  significantly  more  complex  than  simple  gating  association  and  “nearest-neighbor”  correlation  techniques.  Probabilistic  Data  Association(PDA) 
offers  advantages  in  some  applications.  This  paper  examines  two  applications  which  illustrate  the  improvements  that  PDA  techniques  can  offer.  The  first 
example  applies  PDA  to  the  multispectra  1.  kinematic  tracking  problem.  Traditional  techniques  for  correlating  data  from  multiple  sensors  are  insufficient  if  the 
target  density  is  high  and  if  the  sensors  have  very  different  measurement  accuracies.  One  sensor  may  detect  and  establish  tracks  on  multiple  targets  while  a 
sensor  with  lower  resolution  may  detect  and  track  only  one  target.  The  question  is,  which  of  the  multiple  tracks  should  benefit  from  the  other  sensor  s 
measurement?  PDA  offers  a  method  to  resolve  this  question  and  also  to  aid  in  the  formation  and  deletion  of  track  files. 

The  second  example  applies  PDA  to  the  problem  of  correctly  correlating  attribute  measurements  (i.e..  emitter  type,  engine  type,  probability  Friend/Foe/ 
Neutral  etc.)  with  kinematic  data.  Many  times  sensors  that  measure  a  target’s  attributes  give  only  coarse  location  data;  for  example,  an  electronic  support 
measure  (ESM)  system  may  supply  high  confidence  target  identification  (ID),  but  only  coarse  angular  accuracy.  If  there  are  a  number  of  radar  tracks  vnthin 
that  angubr  window,  which  radar  track  should  we  tag  with  the  ESM-generated  ID?  PDA  offers  a  solution  to  this  Attnbute-Kinematic  correlation  problem. 


The  paper  describes  how  to  modify  traditional  fusion  algorithms  (nearest-neighbor  correlation.  Kalman  filter  kinematic  fusion,  and  Bayesian  attribute 
fusion)  to  produce  an  algorithm  that  will  solve  the  two  problems  discussed  above.  It  also  discusses  a  method  for  determining  the  probabilities  required  to 
implement  Probabilistic  Data  Association. 
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1.0  INTRODUCTION 

GE  Aerospace  Electronic  Systems,  located  in  Utica,  New  York,  de¬ 
signs  only  airborne  sensors.  We  operate  under  strict  size,  weight,  and 
speed  constraints,  and  consequently  attempt  to  keep  the  complexity  of 
our  processing  to  a  minimum.  It  is  not  surprising,  then,  that  we  did  not 
immediately  embrace  the  recent  advances  in  target  tracking:  Probabilistic 
Data  Association  (PDA),  Joint  Probabilistic  Data  Association  (JPDA), 
Multiple  Modeling,  etc.  These  algorithms’  complexity  seemed  to  out¬ 
weigh  their  benefits.  As  we  looked  at  the  problem  more  closely  and  as 
our  computational  capability  increased,  we  began  to  see  applications  of 
these  probabilistic  techniques  to  our  airborne  problems.  We  feel  that  the 
multiple  model  filter  improves  our  single-target  fire  control  tracking  accu¬ 
racy,  and  we  are  investigating  the  possibility  of  replacing  our  simpler  ma¬ 
neuver  model  filters  with  the  more  accurate,  more  complex  multiple 
model  technique.  The  following  paper,  however,  discusses  the  application 


of  a  variation  of  probabilistic  data  association  to  the  surveillance/multi¬ 
ple-target  track  problems.  We  have  identified  two  applications  of  PDA 
where  we  feel  the  benefits  may  outweigli  the  increase  in  complexity. 

The  first  and  perhaps  the  most  significant  benefit  occurs  when  we 
apply  PDA  to  the  problem  of  correlating  coarse  ESM  measurements  to 
accurate  radar  tracks  in  the  surveillance  problem  (see  Figure  1). 

Given  that  a  surveillance  radar  has  detected  and  begun  to  track  a 
number  of  targets  and  the  ESM  system  detects  an  emitter  at  a  cenain 
bearing  with  a  large  degree  of  angular  uncertainty,  the  question  is,  which, 
if  any.  of  the  radar  targets  generated  the  detected  emission?  Traditional 
correlation  techniques  (simple  gating  and/or  Chi-Square/Nearest-Neigh- 
bor)  would  tag  the  ESM  measurement  to  the  most  likely  target,  the  target 
closest  to  the  reported  bearing.  In  fact,  it  is  highly  probable  that  some 
other  target  within  the  beam  generated  the  emission.  Tagging  the  ESM 
measurement  to  the  inconect  radar  track  results  in  an  incorrect  ID.  Once 
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we  have  incorrectly  identified  a  particular  target,  it  is  very  difficult  to 
correct  the  mistake  without  complicated  and  extensive  lo^c.  Because  of 
the  coarseness  of  the  ESM  system’s  bearing  measurements,  the  emitter 
ID  itself  is  used  in  the  correlation  process.  If  we  identified  a  particular 
radar  track  as  a  specific  emitter  and  we  receive  ESM  reports  of  that  same 
emitter,  then  those  reports  are  immediately  correlated  to  the  identified 
track.  This  process  actually  reinforces  incorrect  ESM  correlations  and 
could  result  in  high  confidence,  but  completely  incorrect  identification. 
Thus,  we  must  exercise  caution  when  correlating  ID  information  to  as  yet 
unidentified  tracks.  The  Probabilistic  Data  Association  algorithm  exer- 
cises  the  required  caution.  It  allows  us  to  probabilistically  correlate  a  sin¬ 
gle  ID  measurement  to  a  number  of  tracks.  Then  over  time  the  correct 
track  receives  the  majority  of  the  ID  measurement  information,  and  we 
can  correctly  identify  the  track  with  a  high  degi-ee  of  confidence. 

Another  application  in  which  the  benefits  of  PDA  may  outweigh  its 
computational  burden  is  the  multiple-track  and  raid  cell  assessment 
problem.  Figure  2  illustrates  a  generic  tracking  situation  where  PDA  of¬ 
fers  improved  performance.  Assume  that  our  multisensor  platform  is  op¬ 
erating  in  a  low-probability-of-intercept  mode,  and  our  primary  surveil¬ 
lance  sensor  is  an  Infrared  Search  and  Track  (IRST).  The  IR  system 
detects  one  target  and  initializes  a  track.  To  improve  the  range  accuracy 
of  the  track,  we  cue  the  radar  system  to  briefly  track  the  target.  Where 
the  IR  system  detected  only  one  target,  the  radar  system  now  resolves 
three.  Once  the  radar  has  provided  range  for  the  three  targets,  it  breaks 
lock  and  reverts  back  to  passive  operation.  The  composite  tracks  on  each 
of  the  three  targets  are  now  very  accurate  in  both  range  and  angle.  As  the 
targets  close,  the  IR  sensor  continues  to  detect  only  one  target.  Without 
extra  logic,  the  Chi-Square/Nearest  Neighbor  association  algorithm  will 
correlate  the  single  IR  measurement  to  only  the  most  likely  of  the  three 
tracks.  It  is  a  good  possibility  that  the  other  two  tracks  will  be  terminated 
due  to  lack  of  data.  When  the  targets  approach  closely  enough  for  the  IR 
system  to  resolve  them,  the  multisensor  system  must  reinitialize  the  two 
dropped  tracks.  Because  the  only  measurements  available  for  these  two 
new  tracks  are  infrared  azimuth  and  elevation,  their  range  estimates  will 
be  very  poor.  So  poor,  in  fact,  that  they  may  not  be  included  in  the  same 
raid  cell! 

Probabilistic  Data  Association  offers  a  solution  to  this  problem  that  is 
an  alternative  to  solving  the  problem  with  a  logic  tree  that  continually 
assesses  the  number  of  targets  associated  with  each  measurement.  Once 
multiple  tracks  are  established,  PDA  distributes  a  single  measurement  to 
multiple  tracks.  In  our  simple  example,  where  all  three  tracks  are  equally 
likely  to  have  generated  the  single  IR  measurement,  each  track  will 
probabilistically  receive  a  portion  of  the  measurement.  No  tracks  will  be 
terminated  due  to  lack  of  data,  and  as  a  result,  when  the  IR  sensor  finally 
resolves  the  targets,  they  will  still  be  in  the  same  raid  cell  with  fairly  accu¬ 
rate  range  estimates. 
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We  feel  that  these  benefits  alone  warrant  a  closer  look  at  probabilistic 
data  association  techniques.  The  next  step  is  to  develop  an  algorithm  that 
allows  us  to  apply  these  techniques  to  the  ESM/Radar  correlation  prob¬ 
lem  and  to  the  multiple  track/raid  cell  problem. 


2.0  PROBABILISTIC  CORRELATION  OF  ONE  MEASUREMENT 

TO  MULTIPLE  TRACKS  (PCOMMT) 

Probabilistic  Data  Association  assumes  one  target  of  interest  and  mul¬ 
tiple  measurements.  Joint  PDA  assumes  multiple  targets  and  multiple 
measurements.  [Ref.  1]  For  our  applications,  it  is  very  difficult  to  have 
two  or  more  measurements  valid  at  exactly  the  same  time.  As  a  result,  it 
is  possible  to  sequentially  order  the  measurements  in  time  and  process 
them  individually.  Then  our  association  problem  becomes  the  probabilis¬ 
tic  correlation  of  one  measurement  to  the  multiple  tracks  already  estab¬ 
lished.  Figure  3  illustrates  the  problem.  We  must  correlate  the  single 
measurement  to  the  most  likely  track  or  tracks  that  generated  it.  We 
need  to  develop  an  algorithm  that  will  allow  us  to  probabilistically  corre¬ 
late  one  measurement  to  multiple  tracks  (PCOMMT) .  Once  the  correla¬ 
tion  is  complete,  we  must  fuse  the  kinematic  and  attribute  data  in  the 
measurement  vector  to  the  correlated  track  file.  Figure  4  illustrates  the 
necessary  procedure  given  a  one-measurement-to-one-track  file  corre¬ 
lation  scheme.  The  Association/Correlation  function  selects  the  most 
likely  measurement-to-track  correlation.  The  Kinematic  Fusion  function 
employs  a  Kalman  filter  to  fuse  the  kinematic  portion  of  the  measure¬ 
ment  vector  to  the  kinematic  portion  of  the  correlated  track  file.  The 
Attribute  Fusion  function  then  fuses  the  remaining  attribute  portion  of 
the  measurement  vector  to  the  attribute  portion  of  the  state  vector.  (At 
GE  Utica,  we  apply  a  Bayesian  algorithm  to  execute  the  attribute  fusion.) 
In  our  problems,  we  want  to  correlate  one  measurement  to  multiple 
tracks.  Each  one  of  the  functions  pictured  in  Figure  4  requires  modifica¬ 
tion. 


The  Association/Correlation  function  requires  the  least  modification 
of  all  the  functions  pictured.  Figure  5  illustrates  a  more  detailed  block 
diagram  of  the  Association/Correlation  algorithm.  After  passing  a  series 
of  gating  tests,  the  candidate  targets’  “statistical  distances"  from  the 
measurement  vector  are  calculated.  The  track  whose  statistical  distance  is 
a  minimum  is  the  track  that  most  likely  generated  the  measurement  and 
it  is  the  track  that  is  said  "to  correlate"  with  the  measurement  vector.  If 
we  want  to  correlate  the  measurement  with  multiple  tracks,  then  we  select 
a  small  number  of  tracks  that  most  likely  could  have  generated  the  mea¬ 
surement.  That  is,  the  correlated  tracks  are  those  three  to  ten  tracks  that 
have  the  smallest  Chi-Square  numbers  or  statistical  distances  to  the 
measurement  vector.  Thus,  to  change  a  Nearest-Neighbor  correlation  al¬ 
gorithm  to  a  PDA-type  algorithm,  we  need  only  to  allow  more  than  one 
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Figure  2.  Ambiguous  Target  Track 
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Figure  3.  Which  Track  Vector(s)  Should  Z  Update? 
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Figure  4.  Functional  Diagram  of  Measure  me  nt-to-Track  Fusion 
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correlation  per  measurement  to  pass  the  Nearest-Neighbor  test.  The 
problem  is  that  once  we  have  decided  to  accept  multiple  correlations, 
how  do  we  modify  the  Kinematic  and  Attribute  Fusion  functions  so  as  to 
properly  distribute  the  correlated  measurement  across  the  multiple 
tracks?  Figure  6  illustrates  the  modifications  that  we  must  develop.  First, 
let  us  examine  the  modification  to  the  Kinematic  Fusion  function. 

2.1  Modification  to  Kinematic  Fusion 

Our  goal  is  to  use  one  measurement  to  probabilistically  update  many 
kinematic  track  files.  Each  kinematic  state  estimate  is  generated  by  its 
own  unique  Kalman  filter.  We  must  find  a  method  to  modify  the  filter  to 
account  for  the  fact  that  we  are  distributing  the  measurement  across  mul¬ 
tiple  tracks.  One  form  of  the  discrete  Kalman  filter  is 


%  (k/k-1)  =  O  •  Xi  (k-l/k-1)  +  B  •  U(k-l)  (1) 

Pj  (k/k-1)  =  <I>  •  Pi  (k-l/k-1)  •  +  Qj  (k-1)  (2) 

Ki  (k)  =  Pj  (k/k-1)  *  HjT  .  [Hj  •  Pj  (k/k-1)  •  HiT  +  R(k)]"l  (3) 

Pj  (k/k)  =  [I  -  Kj  (k)  •  Hj  1  .  Pi  (k/k-1)  (4) 

^  (k/k)  =  Xi  (k/k-1)  +  Ki  (k)  •  lz(k)  -  H]  •  Xi  (k/k-1))  (5) 


where  Xi  (k/k)  is  the  kinematic  state  vector  estimate  of  target  i  valid  at 

time  tj^  given  data  up  to  and  including  data  valid  at  tj^.  Xj  (k/k-1)  is  the 
estimate  of  the  same  state  vector,  but  only  using  data  up  to  time  tj^.j .  O 
is  the  state  transition  matrix;  BU(k-l)  is  the  deterministic  forcing  func¬ 


tion  (ownship  motion);  Pj  (k/k)  is  Xj  (k/k)’s  error  covariance  matrix, 
Q-  (k-1)  is  the  state’s  process  noise  covariance  matrix;  is  the  linear 
transformation  from  state— space  to  the  measurement— space;  Kj  (k)  is  the 
Kalman  gain  matrix;  and  R(k)  is  the  measurement  noise  covariance  ma¬ 
trix  associated  with  the  measurement  vector  Z  (k) .  This  notation  empha¬ 
sizes  the  fact  that  equation  1  updates  the  kinematic  state  estimate  to  time 
based  only  on  past  data.  Equation  5  then  corrects  that  prediction  with 
the  current  measurement.  The  point  is  that  if  no  measurement  correlated 
to  track  i  at  time  tj^,  then  we  update  track  i  to  time  tj^  with  equation  1 
only.  If.  however,  the  measurement  did  correlate  to  track  i,  then  we  must 
go  one  step  further  and  correct  the  track’s  estimate  with  equation  5.  We 
can  use  these  facts  to  modify  the  Kalman  filter  to  account  for  PCOMMT. 

If  we  know  that  track  i  did  nQt  generate  the  current  measurement. 

'Z(k),  then  we  should  update  its  estimate.  Xi  (k-l/k-1),  to  time  tj^  with 
equation  1;  i.e,, 

Xi  •  (k/k-1)  =  (p  .  ^  (k-l/k-1)  +  B  .  Li(k-l)  (6) 

where  we  have  included  the  prime  (’)  on  X  to  indicate  that  this  is  an 
intermediate  result,  and  where  (j)  spans  the  time  from  t|^_i  to  t^.  But  if 
track  i  generate  the  measurement,  then  we  should  correct  this  pre¬ 
dicted  estimate  with  equation  5,  i.e., 

^  '(k/k)  =  Xi  '(k/k-1)  +  Ki  (k)  .  lZ(k)  -  Hj  Xi  '(k/k-l)].  (7) 


CORRELATED 

track  FILE(S) 


UPDATED 
COMPOSITE 
TRACK  FILE(S) 


Figure  6.  Kinematic  and  Attribute  Fusion  Modifications 
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Wc  do  not  know,  however,  which  case  is  true.  Therefore,  let  us  treat 
each  case  separately  and  then  combine  them  probabilistically.  Assuming 
for  a  moment  that  given  the  measurement  Z  (k) .  we  know  the  probability 
that  the  ith  track  generated  it,  P(^  /Z).  we  can  combine  the  two  update 
possibilities  (equations  6  and  7)  with  the  Total  Probability  Theorem. 

^  ^  —  _A. 

Xi  (k)  =  '(k/k)  .  P(Xi  /  Z)  +  Xi  '(k/k~l)  •  P(Xi  /  Z)  (8) 

where  P(^  /Z)  is  the  probability  that  given  ”Z(k),  the  ith  track  did  not 
generate  the  measurement.  If  we  knew  P(Xi/Z),  then  we  could  use 
equation  8  to  distribute  a  single  measurement  to  multiple  tracks.  Before 
attempting  to  derive  an  expression  for  P(Xj  /Z),  let  us  try  to  simplify 
equation  8. 

First,  either  Xi  did  help  generate  "zCk),  or  it  did  not.  There  are  no 
other  possibilities.  Therefore, 

PC^/Z)  -  1.0-P(^/Z).  (9) 


Substituting  this  expression  for  P(Xi/Z) 
Xi  '(k/k)  (equation  7)  into  equation  8, 

%  (k)  =  {Xi  '(k/k-1)  +  Ki  (k)  .  [Z(k)  -  Hi 


+  Xi'(k/k-l)  •  [1  -P(Xi/Z)] 


and  the  expression  for 


Xi'(k/fc-l)l}.P(l^/Z) 


(10) 


Simplifying  equation  10  and  dropping  the  prime  notation  on  Xi'(k/k-l) 

because  Xi'(k/k-l)  =  Xi(k/k-l),  we  get  an  expression  for  the  compos¬ 
ite  estimate  of  the  ith  target 

^  ^  -k. 

Xj  (k)  =  (k/k-l)  +  Kj  (k)  •  P(Xi  /  Z) 

.[Z(k)  -Hi^(k/k-l)].  (11) 


This  result  is  identical  to  the  normal  estimate  correction  equation  5 
except  that  the  gain  is  scaled  by  the  probability  that  Xi  generated  the 
measurement  Z(k).  If  P(Xi  /Z)  equals  one  (i.e.,  we  are  certain  that 
generated  Z  (k)),  then  the  modified  algorithm  reduces  to  the  normal  Kal¬ 
man  filter.  If,  however,  we  are  cenain  that  Xi  did  iM  generate  'Z(k). 
then  P(Xi  /Z)  equals  zero  and  the  measurement  fails  to  influence  the  ith 
track  estimate.  The  effect  of  P(Xi  /Z)  is  to  adjust  the  bandwidth  of  the 
filter  based  on  our  certainty  that  %  generated  the  measurement.  The 
more  uncertain  we  are,  the  lower  the  bandwidth  to  measurement  'z(k). 
This  is  exactly  the  behavior  we  desire  for  our  modified  filter.  We  want  the 
measurement  to  influence  the  most  likely  tracks  the  most  and  the  least 
likely  tracks  the  least.  All  that  remains  to  complete  our  modification  to 
the  kinematic  fusion  function  is  to  determine  how  the  one  measurement/ 
multiple  tracks  algorithm  (PCOMMT)  affects  the  other  three  filter  equa¬ 
tions  (#2  through  #4). 

Since  we  are  still  propagating  the  state  estimate  over  the  interval 
(*k-l*  *k)’  process  noise  errors  and  propagation  errors  accounted  for 
in  equation  2  still  occur.  Equation  2  remains  unchanged.  However,  since 
we  have  attenuated  the  gain  by  P(Xi  /Z),  we  lose  some  of  the  effect  of 
the  measurement  on  the  state  estimate.  The  error  is  not  reduced  as  much 
as  it  would  have  been  previously.  The  full  error  reduction  capability  of 
the  gain  remains  unrealized  and  the  error  correction  equation  4  is  incor¬ 
rect.  We  must  scale  the  gain.  Kj  (k).  in  equation  4  by  P(Xi  /Z)  to  reflect 
the  actual  correction  we  are  performing  in  equation  11.  The  result  is  that 
we  can  achieve  the  correct  modification  to  equations  4  and  5  by  scaling 
the  gain  equation  3.  The  resulting  Kinematic  Fusion  algorithm  is 


Xj  (k/k-l) 

=  <p  •%  (k-i/k-i: 

)  +  B  •■u(k-l) 

(12) 

Pj  (k/k-l) 

=  ^  •  Pj  (k-l/k-l) 

•  +  Qj  (k-1) 

(13) 

Kj  (k)  = 

P(^  /z) 

•  Pj  (k/k-l)  •  Hj'T 

•  [Hj  Pj  (k/k-l) 

HiT  +  R(k))-1 

(14) 

Pj  (k/k)  = 

[I  -  Kj  (k)  .  Hj  ].  .  1 

-j  (k/k-l) 

(15) 

%  (k/k)  = 

Xi  (k/k-l)  +  Kj  (k) 

•  [Z(k)  -  Hj  • 

Xi  (k/k-l)] 

(16) 

Comparing  this  algorithm  with  equations  1  through  5,  we  see  that 
PCOMMT  requires  only  a  minor  modification  to  the  traditional  Kalman 
filter. 


2.2  Attribute  Modifications 

GE  Utica’s  approach  to  Attribute  Fusion  has  been  to  employ  the 
Bayesian  recursive  algorithm.  This  technique  requires  the  selection  of  a 
complete  set  of  mutually  exclusive  attributes  such  as  the  possibilities  of  a 
target  being  either  Friend,  Foe,  or  Neutral.  Once  we  have  selected  an 
attribute  set,  we  maintain  the  probability  of  each  element  in  the  set  being 
correct.  Because  the  set  is  mutually  exclusive, 

P(Fr)  +  P(Fo)  +  P(N)  =  1.0  (17) 

Now  let  us  assume  that  at  time  tj^_j  we  know  the  probability  that  the 
ith  target  is  a  Foe,  Pj  (Fq)j^_j.  and  we  receive  a  measurement.  Zj^, 
containing  attribute  data,  Z^j^,  at  time  tj^.  How  docs  the  traditional 
Bayesian  algorithm  estimate  the  probability  that  the  ith  target  is  a  Foe 
based  on  this  new  information?  What  is  Pj  (Fq)j^?  The  algorithm  em¬ 
ploys  both  Bayes  Theorem  and  the  total  Probability  Theorem  to  find 
Pj  (pQ^k  ■  Based  on  a  priori  knowledge,  we  can  estimate  the  probability 
density  functions  pj  (Zgi^lFo),  Pj  (Zgi^lFr),  and  pj  (Z^j^jN)  for  the  at¬ 
tribute  clement  of  measurement  "Zi^.  Then  if  we  let  the  a  priori 
probability  of  target  i  being  a  Foe  equal  Pj  (Fq)j^_i,  we  can  use  Bayes 
Theorem  and  the  Total  Probability  Theorem  (TPT)  to  write  [Ref  2] 

p.(Po),=  _ PL^KlF°)«Pi(F°)k-, _ 

(PjCZg^lFr)  •Pi(Fr)k.i  +Pi(Zak|Fo).Pi(Fo)k.i 

+  Pi(ZaklN)»Pi(N)k_il  (18) 

where  Pj  (Fq)|^  is  really  a  conditional  probability  conditioned  on  the 
data,  but  for  brevity  we  have  dropped  the  condition. 

If  we  let 
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Pi(Z3k|f)  .  Pi  (0  k-1  =  Pi(Z3klFr)  .  Pj  (Fr)  k-, 
+  Pi(2aklFo)»Pi(Fo)k_i 
+  Pi(Z3k|N)*Pi(N)k_i 

then  we  can  write 


Pi(Fo)k  = 


Pi(ZaklFo)»Pi(Fo)k-i 

i  Pi(Z3,,|0  .Pi(f)  k-1 

F=1 


(19) 


(20) 


This  is  the  traditional  Attribute  Fusion  technique:  employ  the  previous 
probability  and  the  known  density  functions  to  calculate  the  new  estimate 
of  Pj  (Fq)j^.  Each  element  of  the  attribute  set  must  be  updated  every 
time  we  receive  a  new  measurement. 


The  traditional  method  assumes  that  the  ith  target  generated  the 
measurement,  Zj^.  It  assumes  perfect  correlation.  In  reality,  the  update 
equation  should  be  conditioned  on  this  assumption.  That  is.  given  that 
Xi  generated  the  measurement,  Zj^,  the  probability  of  Xj  being  a  Foe  is 


Pi(ZaklF°-Xi).Pi(Fo)k-i 

2  Pi(Z3k|f.Xi)»Pi(f)k-l 

F-1 


(21) 


The  PCOMMT  algorithm  does  noi  make  the  perfect  correlation  as¬ 
sumption  in  its  Attribute  Fusion  function  so  it  must  correct  the  traditional 
Bayesian  estimate  of  Pj  (Fq)j^  .  If  we  use  the  traditional  method  to  calcu¬ 
late  Pj  (Fo/  Xj  )j^  ,  then  we  can  correct  this  estimate  for  PCOMMT  with 
the  TPT  and  knowledge  of  P(Xj  |Z).  The  corrected  probability  is 

Pi(Fo)k  =Pi(Fo|Xi)^.  P(Xj|Z)  ^22) 

-h  Pi(Fo|Xj)^.  P(Xi|Z) 

Now  the  probability  of  the  ith  track  being  a  Foe,  given  that  it  failed  to 

generate  a  measurement  at  time  tj^,  Pj  (Fq|  Xj  )j^.  has  to  equal  the  previ¬ 
ous  probability  of  Foe  estimate.  In  the  absence  of  new  data  we  have  to 
keep  Pj  (Fq)  constant, 
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Pi{Fo|Xi)|^=  Pi{Fo)k-l 

and  as  before  either  Xi  generated  or  it  did  not. 

P(Xi|Z)  =  tl  -  P(XilZ)l 

We  can  write  our  correction  equation  22  as 

Pi(Fo)k  =  Pi(P°lXi\  *  _  (25) 

+  Pi(Fo)k-l  •  n  “  P(Xi|Z)] 

As  in  the  kinematic  fusion  case,  we  have  attenuated  our  estimate  ajid 
added  in  the  possibility  that  perhaps  no  update  should  occur  because  Xi 
may  not  have  generated  Zk-  The  result  is  a  slower,  more  cautious  reac¬ 
tion  to  the  incoming  data  and  the  “L^-^iss"  analogy  applies  here  as 
well.  The  modification,  in  terms  of  P(Xi  IZ).  is  given  by 

Pi(Fo)k  =  |Pi(Fo|Xi)|^  -  Pi(Fo)k^_i)  •  P(XilZ) 

+  Pi(Fo)|,_i 

Both  equations  25  and  26  are  equivalent  expressions  for  Pj  (Fq)];- 
They  illustrate  that,  although  the  modification  is  not  as  simple  as  in  the 
Kinematic  Fusion  case,  it  is  a  minor  addition  to  the  tradition^  Bayesian, 
attribute  fusion  approach.  Once  we  have  determined  P(XilZ),  the 
modifications  require  two  more  adds  and  one  more  multiply  per  attribute 
set  element.  Thus,  the  PCOMMT  algorithm  is  not  only  a  better  method 
for  correlating  attribute  data  to  kinematic  data,  but  it  also  appears  to  be  a 
feasible  solution  to  the  overall  correlation  problem  assuming  we  can  cal¬ 
culate  P(Xi  Iz). 

The  exact  calculation  of  P(  Xi  I"?)  is  nontrivial.  To  account  for  all  of 
the  possibilities,  computation  of  P(Xi  \  Z)  is  extremely  complex, 

3.0  DERIVATION  OF  THE  PROBABILITY  THAT  TRACK| 
GENERATED  THE  MEASUREMENT,  P(Xi|Z) 

To  find  P(Xi  I'z).  we  must  know  all  of  the  possible  tracks  and  com¬ 
binations  of  tracks  that  could  have  generated  Z^-  We  must  find  the 
probability  of  each  possibility  being  correct.  Theji  w^can  sum  all  of  the 
possible  probabilities  that  include  Xj  to  find  P(Xi  \  Z).  The  first  step  is 
to  define  all  possible  ways  of  generating  Z^- 

3.1  Possibilities 

The  first  and  most  likely  possibility  is  that  one  and  only  one  target  in 
the  proximity  of  the  measurement  generated  Z^*  (Note:  We  consider 


false  alarms  valid  targets  until  we  prove  otherwise.)  The  second  possibility 
is  that  two  and  only  two  targets  combined  to  generate  the  measurement. 
The  third  possibility  is  that  three  and  only  three  targets  combined  to  gen¬ 
erate  the  measurement.  We  could  go  on  and  on;  however,  the  possibility 
of  more  than  three  targets  combining  to  generate  Z^  ‘S  extremdy 
likely,  and  the  inclusion  of  the  possibility  into  the  derivation  for  P(  Xj  ( Z) 
significantly  increases  its  complexity.  So  we  assume  that  if  there  are  more 
than  three  targets  present,  they  cannot  group  themselves  so  closely  that 
our  sensors  will  fail  to  discriminate  them.  In  addition,  we  make  the  as¬ 
sumption  that  no  new  target  (a  category  that  includes  false  alarms  as  well 
as  valid  targets)  will  combine  with  other  targets  to  generate  the  measure¬ 
ment.  New  targets  act  alone.  In  effect,  this  assumption  means  that  we  will 
ignore  the  possibility  that  a  new  target  might  combine  with  an  existing 
target  to  generate  the  measurement.  If  this  situation  actually  occurs,  then 
we  will  give  the  entire  measurement  to  the  established  tracks.  With  these 
assumptions.  Figure  7  illustrates  the  possible  events.  (Note  that  although 
the  targets  themselves  generate  the  measurements,  we  have  shown  track 
positions  as  estimates  of  the  target  state  vector  because  that's  all  we  have 
to  go  on.)  Figure  7  shows  that  we  ignore  targets  outside  the  Association’s 
“Fine  Gate”,  which  limits  our  algorithm  to  a  small  number  of  targets,  and 
we  choose  to  perform  PCOMMT  on  the  three  to  five  targets  with  the 
lowest  Chi-square  numbers.  It  also  illustrates  the  possible  combinations 
of  the  three  targets  near  the  measurement  that  may  have  coinbmed  to 
generate  Zk-  Now  we  must  derive  the  probabilities  that,  given  Zk-  each 
one  of  the  events  pictured  could  have  generated  the  measurement. 

3.2  Probabilities 

To  illustrate  the  complex  probabilities  associated  with  this  problem, 
we  start  with  the  symbolic  Venn  diagram  shown  in  Figure  8.  Now  it  be¬ 
comes  apparent  why  we  broke  the  possibilities  down  imp  three  distinct 
groups  in  Figure  7.  Figure  8  shows  that  all  of  the  events  ocwrring  within 
one  of  the  three  groups  are  mutually  exclusive.  That  is,  if  Xi  generated 
the  measurement  alone,  then  none  of  the  other  events  in  Group  1  could 
have  occurred.  Xi,  X3.  X5  ,  etc.,  played  no  part  in  generating  the  n^a- 
surement.  Or  similarly,  given  that  one  and  only  one  target  generated  Zk- 
the  sum  of  all  the  conditional  event  probabilities  within  Group  1  must 
equal  one.  With  this  knowledge,  we  can  write  all  of  the  possibilities  as 
shown  in  Table  1. 

Table  1  clarifies  the  probabilities  we  must  derive:  The  single 
target  probabilities,  P(Xi  IZ.l);  the  doub^  mget  probabilities, 
P(Xi&Xj  I'z. 2):  the  new  target  probability.  P(XntlZ.l):  and  the  group 
probabilities.  P(l),  P(2).  and  P(3).  Some  of  these  derivations  are  com¬ 
plex,  some  require  sweeping  assumptions,  and  some  may  result  in  negligi¬ 
ble  probabilities.  So  before  we  continue,  let  us  discuss  the  meaning  of 
some  of  these  terms.  The  most  questionable  terms  in  terms  of  significance 
are  those  accounting  for  the  possibility  of  multiple  targets  combining  to 
generate  a  common  measurement. 

Why  account  for  the  possibility  of  more  than  one  target  combining  to 
generate  a  single  measurement?  Is  this  an  unlikely  possibility?  The  answer 


THAT  MAY  HAVE  COMBINED  - 
TO  GENERATE  MEASUREMENT 


POSSIBLE  CAUSES  OF  MEASUREMENT  Zk  | 

GROUP  1: 

Xi  ALONE 

X2  ALONE 

X3  ALONE 

X4  ALONE 

X5  ALONE 

NEW  TARGET  ALONE 

GROUP  2: 

X1  ANDX2 

Xi  AND  X3 

X2  AND  X3 

GROUP  3:^ 

Xi  ANDX2  ANDX3 

Figure  7.  Possible  Contributors  to  Measurement 
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GROUP  1:  ONE  AND  ONLY  ONE 
GROUP  2:  TWO  AND  ONLY  TWO 
GROUP  3:  THREE  AND  ONLY  THREE 


Figure  8.  Venn  Diagram  of  Possibilities 


TABLE  L  EVENT  PROBABILITIES 


P{X}  ALONE!  2) 

-  GIVEN  ?k.  THE  PROBABILITY  THAT 

THE  Hh  TRACK  ALONE  GENERATED 

IT. 

-  GVEN  THAT  ONE  TARGET 

GENERATED  ?k.  AND  GIVEN  THIS 

PARTICULAR  ?k.  THE  PROBABILITY 
THAT  THE  GENERATING  TRACK 

WAS  THE  ith  TRACK. 

GROUP  1: 

P(1) 

-  PROBABILITY  THAT  ONE  TARGET 

ALONE  GENERATED  ?k. 

P(Xi  ALONE|2)  ^  P{Xi|'Z,1)  •  P(1) 

P{X2  ALONEt"?)  =  P{X2|Z,1)  •  P(1) 

P(X3  ALONE!?)  =  P{?3|?.1)  •  P(1) 

P(X4AL0NE|?)  =  P(X4|?.1)  •  P(1) 

P(X5  ALONEI?)  =  P(X5|?,1)  •  P(1) 

P(Xn|  ALONEIZ)  =  P{Xnt|  2  ,1)  •  P{1) 

2  P(Xi|Z.1)  +  P{Xnt|Z,1)  =  1 

N1 

P{Xnt  ALONE!?) 

P(Xnt|?.1) 

P(Xi  &  Xj  ONLY!?) 

-  GIVEN  ?k,  THE  PROBABILITY  THAT 

THE  NEW  TARGET  ALONE 

GENERATED  IT. 

-  GIVEN  THAT  ONE  TARGET 

GENERATED  ?k,  AND  GIVEN  THIS 

PARTICULAR  ?k,  THE  PROBABILITY 
THAT  THE  GENERATING  TRACK 

WAS  A  NEW  TARGET. 

-  GIVEN  ?k.  THE  PROBABILITY  THAT 

GROUP  2: 

TWO  TARGETS,  Xj  AND  Xj, 

COMBINED  TO  GENERATE  IT, 

P(Xi  &  X2  ONLYjZ)  =  P{Xi  &  X2|?.2)  •  P(2) 

P(Xi  &  X3  ONLY|Z)  =  P(Xi  &  X3|?.2)  •  P(2) 

P(X2  &  X3  ONLYI  Z)  »  P(X2  &  X3|  Z.2)  •  P{2) 

1  2  P(X2&  X3|Z,2)  =  1 

i=1  j=i+1 

GROUP  3: 

P(Xi  &  Xjt?,2) 

P(2) 

-  GIVEN  THAT  TWO  TARGETS 

GENERATED  ?k,  AND  GIVEN  THIS 

PARTICULAR  ?k.  THE  PROBABILITY 
THAT  THE  GENERATING  TRACKS 

WERE  TRACK  I  AND  TRACK  j. 

-  PROBABILITY  THAT  TWO  AND  ONLY 
TWO  TARGETS  COMBINED  TO 
GENERATE  ?k. 

P(Xt  &  X2  &  X3I?)  =  P(Xi  &  X2  &  X3|?,3)  •  P(3) 

P(Xi  &  X2  &  X3|?,3)  =  1 

P{Xi  &  X2  &  X3I?) 

-  GIVEN  ?k,  THE  PROBABILITY  THAT 

THE  THREE  TARGETS  COMBINED 

TO  GENERATE  THE  MEASUREMENT. 

P{Xi  &  X2  &  X3|?.3) 

-  GIVEN  THAT  THREE  TARGETS 

GENERATED  ?k.  AND  GIVEN  THIS 

PARTICULAR  ?k,  THE  PROBABILITY 
THAT  THE  GENERATING  TRACKS 

WERE  TRACK  1  AND  TRACK 2  AND 
TRACK3. 

P{3) 

-  PROBABILITY  THAT  THREE  AND 

ONLY  THREE  TARGETS  COMBINED 

TO  GENERATE  ?k. 
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is  yes  and  no.  If  the  sensor  measures  unique  attribute  data  (e.g..  trans¬ 
mitter  frequency),  then  no  two  targets  will  ever  combine  to  generate  a 
common  measurement,  and  the  answer  is  yes.  the  multiple  target  prob¬ 
abilities  are  negligible.  If,  however,  the  measurement  is  not  necessarily 
unique,  then  the  multiple  target  probabilities  are  nonnegligible.  For  ex¬ 
ample,  assume  that  an  infrared  sensor  has  established  two  tracks  very 
close  together  in  angle  (so  close  that  our  radar  dwells  on  both  targets 
during  a  single  coherent  pulse  interval  (CPI)) .  If  the  targets  are  very  close 
in  range,  then  the  radar  will  receive  only  ONE  detection.  Which  track  file 
should  get  the  benefit  of  this  single  radar  measurement?  Since  both  com¬ 
bined  to  generate  the  track,  then  both  should  receive  the  entire  Z^.  The 
nearest-neighbor  technique  would  give  the  entire  measurement  to  only 
one  of  the  tracks.  The  PCOMMT  algorithm  will  distribute  the  measure¬ 
ment  across  both  tracks,  but  with  what  weighting  function?  If  we  ignore 
the  possibility  of  more  than  one  target  gen  gating  a  single  measurement, 
then  the  best  value  we  could  achieve  for  P( Xj  [  Z)  is  0.5  (in  other  words, 
give  half  the  measurement  to  each  track) .  But  this  is  not  anywhere  near 
what  the  tracks  should  have  received.  They  both  combined  to  generate 
the  radar  measurement  so  they  should  both  receive  the  gntjrg.  measure¬ 
ment.  ?(%  I Z)  should  equal  1  for  both  tracks.  Since  the  possibilities  of 
each  target  having  generated  the  measurement  alone  are  mutually  exclu¬ 
sive,  unless  we  include  the  possibility  of  more  than  one  target  combining 
to  generate  "Zk-  we  can  never  give  the  entire  measurement  to  more  than 
one  target.  Thus,  the  multiple  target  probabilities  are  nonnegligible  in  this 
case.  They  are  particularly  valuable  when  dealing  with  kinematic  mea¬ 
surements  from  a  sensor  suite  containing  widely  varying  degrees  of  indi- 
^dual  sensor  accuracy.  The  derivation  for  the  multiple  target  probabili¬ 
ties  is  extremely  tedious.  We  present  the  derivation  for  single  target  prob¬ 
ability  because  it  applies  directly  to  the  ESM/Radar  correlation  problem. 
But  we  present  only  a  brief  derivation  for  the  remaining  probabilities  in 
Table  1. 

For  the  ESM/Radar  correlation  problem,  each  measurement  is 
unique;  it  could  have  only  come  from  one  target.  All  we  need  to  calculate 
from  Table  1  is  the  Group  1  or  single  target  probabilities.  For  this  appli¬ 
cation. 


Theorem  and  the  Total  Probability  Theorem  again  allow  us  to  determine 
an  a  posteriori  from  the  a  priori  pdf. 


P(Xi|Z.l) 


_ P(ZiXi.l)  >P(3^) _ 


where  the  P(  Xn)  ’s  are  r^e  probabilities  that  a  particular  Xn  will  generate 
a  measurement,  and  T  is  the  number  of  tracks  that  qualifies  for 
PCOMMT.  The  probability  that  a  target  will  generate  a  measurement  is 
equal  to  its  probability  of  detection,  which  w^can  approximate  from  the 
track’s  range  estimate.  We  emphasize  that  P(Xn)  is  the  nth  track’s  prob¬ 
ability  of  detection  by  adding  the  subscript  d.  Rewritten,  our  equation  for 
the  probability  of  the  ith  track  having  generated  the  current  measure¬ 
ment,  and  the  cornerstone  of  the  PCOMMT  algorithm  is 


P(Xi|Z.l)  = 


p(Z|Xi,l)  »Pri(Xi) 


■  T 

2 

n=l 


-  (32) 


2  [p(^X„,l)  TdtXn)]  +  P(^X„,.l)  TdlXm) 


where  everything  is  known  in  equation  32  except  the  new  target  terms. 
We  need  some  reasonable  values  for  them. 


3.3.1  New  Target  Probabilities 

Attaching  a  value  to  the  probability  of  a  new  target  appearing  is 
guesswork  at  best.  But  since  an  ESM  system  generally  has  a  much  greater 
range  than  the  radar  system  it  augments,  the  possibility  of  new  tracks  is 
very  real,  and  we  had  better  make  our  guesswork  as  accurate  as  possible. 

Let  us  take  the  first  term,  P(Z|  Xnf^)-  ^  Gaussian  density 

described  by  the  form  of  equation  28.  Note  that  the  equation  employs  the 
state  estimate  as  the  random  variable's  mean  value.  But  since  the  target 
in  question  is  new,  we  have  established  no  estimate,  and  in  fact  if  we  do 

initialize  an  estimate,  its  first  value  will  equal  Z  (k)  so  that 


P(^|Z)  =P(^|Z,l)  (27) 

If  we  calculate  P(Xi  [Z,!),  then  we  have  found  P(Xi  \  Z).  Let  us  derive 
P(Xi  |Z.l). 

3.3  ESM/Radar  Correlation  Application  Single  Target 
Probabilities  -  P(Xi|Z,l) 

Although  there  is  a  lot  of  statistical  information  in  Table  1  that  must 
be  derived,  we  already  know  one  very  essential  piece  of  information.  We 
know  the  probability  density  function  (pdf)  of  the  measurement,  given 
the  track  estimate.  Given  that  the  ith  track  generates  the  measurement 

alone,  then  the  distribution  of  Zj^  around  the  estimate  X\  (k|k-l)  will  be 

Gaussian  with  a  mean  of  Hj  X^  (kjk-1),  and  a  covariance  equal  to  the 
sum  of  the  estimate’s  error  covariance.  Pi(klk-l).  and  the  measurement 
noise  covariance,  R, 

P(ZlXi,l)  =  [(2ir)’^  •|hphT+r|]‘®'^  ^  (28) 

,  e  -0.5  .  [Z-Hxf  •  [HPH  +  R]-'  •  (Z-HXl 

where  the  estimate  and  covariance  is  valid  at  the  measurement  time,  tj^, 
and  M  is  the  number  of  components  in  Z  •  In  terms  of  the  innovation 
(Vj)  and  its  covariance  (Si),  equation  28  becomes 

p(zlXi.l)  =  [(217)M  .  |Sil]'“-^  .  e-»-5  •  •  n  (29) 

or  we  could  write  the  density  function  in  terms  of  the  Chi-square  statistic 
(Xi^)  generated  in  the  Association/Correlation  function 

p(zjXi,l)  =  [(2TT)^  •  |Sil]  •  Xi  (30) 

Equation  30  represents  the  a  priori  pdf.  Given  that  Xi  will  generate  a 
measurement,  equation  30  is  the  probability  distribution  of  it  gei^rating 
'Zjj^.  We  want  the  a  posteriori  probability.  In  other  words,  given  Zj^  oc¬ 
curred,  what  is  the  probability  that  >4  generated  it,  P(Xi  |Z.l)?  Bayes 


Z(k)  -  Hi  Xi  (k|k-l)  =  0 

There  are  other  ways  to  handle  the  new  target’s  estimate,  but  for  our 
case,  let  us  substitute  equation  33  into  the  density  function  equation  28. 
Then  we  have 

P(Z|Xn,  .1)  =  [(217)'^  .  IRnt  +  Rl] 

where  R^t  «  the  new  target  covariance  matrix.  Now  we  can  work  with 
Rj^t  to  make  the  pdf  small  enough  to  prevent  new  tracks  from  stealing 
measurements  from  established  tracks.  If  we  always  initialize  our  tracks 
with  the  measurements,  then  the  estimate’s  error  should  become  less 
than  the  measurement  error.  The  inequality 

H  •  P  •  h'T  <  R 

should  hold  for  tracks  receiving  consistent  measurements 
vative,  let  us  choose 

Rnt  =  4  .  R 

to  account  for  setting  HX(k)  equal  to  Z(k).  and  for  the  possibility  of 
existing  tracks  having  degraded  estimates  due  to  sporadic  measurements. 
Then,  as  a  first  cut, 

p(Zl5?nfl)  =  pTT)^  •  |5Rl] 

which  is  a  constant  over  the  region  [(21T)^  |5R[]  .  centered  on  the 

measurement.  This  is  close  to  a  three-sigma  gate  for  track  initialization. 

The  other  unknown  term,  Pd(Xnt)'  difficult  to  quantify.  It  is  the 
probability  that  new  targets  (including  false  alarms)  will  appear.  It  de¬ 
pends  on  the  environment,  the  mission,  etc.  It  could  conceivably  be  any 
value  between  zero  and  one,  but  remember  the  higher  the  probability  we 
assign  new  targets,  the  lower  the  existing  track  probabilities  will  become. 
Every  application  has  a  different  value  for  Pd(Xnt )  •  It  is  probably  best  to 
start  with  a  low  value  for  new  target  probability  of  detection  and  then 
quantify  Pd(^nt)’^  particular  application. 


(35) 

.  To  be  conser- 

(36) 
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3.4  ESM/Radar  Algorithm  Summary 

This  completes  the  derivation  of  all  the  terms  required  for  the  ESM/ 
Radar  fusion  algorithm.  Because  each  ESM  measurement  is  unique  and 
could  only  originate  from  a  single  target,  the  possibility  of  more  than  one 
target  combining  to  generate  Z(k)  is  zero.  Therefore,  for  this  case,  the 
Group  2  and  Group  3  probabilities  listed  in  Table  1  all  equal  zero  and  the 
probability  that  one  and  only  one  target  generated  "Z.  P(l).  equals  one. 
Thus. 

P(Xi!Z)  =P(Xj|Z.l)  (38) 

and  the  ESM/Radar  hision  process  is  summarized  below: 

•  Association/Correlation 

—  Same  as  traditional  “Nearest-Neighbor”  algorithm  except  in¬ 
stead  of  allowing  only  the  nearest-neighbor  to  pass,  the  cor¬ 
relation  criterion  allows  the  nearest-neighborj  to  pass 

For  EVERY  track  that  passes  the  Association/Correlation  algorithm,  the 
following  Kinematic  and  Attribute  Fusion  functions  must  be  performed 

•  Kinematic  Fusion 


•  Attribute  Fusion  (Friencf/Foe/Neutrol  Example) 


Pi(Fo|Xi)|^  =  - 

Pi(ZaklP°)*Pl(Fo)k-1 

i  P(Zaklf)«Pi(f)k-l 

Pi(Fr|Xi)j^=  ■ 

Pi(ZaklF>').Pi(Fr)k_, 

2  P(Zaklf)*Pi(0k.i 

Pi(N|Xi)^=  1 

-  Pi(Fo|Xi)|^  -  Pi(Fr|Xi) 

Modify: 


Pi(Fo)j^  =  Pi(FolXi)^«  P(Xj)Z)  Pi(Fo)i^_l.P(3q|Z) 
Pi(Fr)k  =  Pi(Fr|Xi)^.  P(3qiZ)-h  Pj  (Fr)  k„i  •  P(^  |Z) 
Pi(N)j^  =  l-Pi{Fo)^  -Pi(Fr)j^  I 


where  P(Xj  |Z)  is  calculated  as  shown  in  Figure  9. 

4.0  MULTIPLE  TARGET  AND  RAID  CELL  ASSESSMENT 
APPLICATION:  MULTIPLE  TARGET  PROBABILITIES 

To  find  P(Xj  jZ)  for  the  situation  where  measurements  are  ambigu¬ 
ous  and  it  is  possible  to  assign  the  entire  measurement  to  a  number  of 
tracks,  we  must  calculate  the  remaining  probabilities  listed  in  Table  1. 
Remember  that  we  have  made  the  assumption  that  at  most,  three  targets 
should  receive  the  entire  measurement.  The  following  derivation  is  tedi¬ 
ous  and  requires  making  a  number  of  assumptions  that  may  or  may  not 
apply  to  the  reader’s  specific  situation.  We  include  the  high  points  and 
results  as  points  of  interest. 

We  begin  by  looking  at  the  Group  2  probabilities  listed  in  Table  1. 
We  must  find  the  probability  of  each  of  the  three  closest  targets  combin¬ 
ing  with  one  of  the  other  two  to  generate  Z(k).  (Remember  that  we  have 

assumed  that  at  most,  three  targets  would  combine  to  generate  Z(k),) 
We  must  find  two  probabilities:  the  probability  that  two  targets  did  gener¬ 
ate  Z(k),  P(2);  and  the  probability  that  given  two  targets  combined  to 
generate  Z(k)  and  given  Z(k),  the  ith  and  jth  targets  were  the  two. 
P(Xj &Xj lZ.2).  Let  us  start  with  ?(%  &Xj  |■z.2). 

Given  that  two  targets  generated  Z(k),  if  we  knew  the  probability 
distribution  of  Z(k)  around  the  ith  and  jth  target.  Xj  and  Xj.  and  if  we 
knew  the  probability  that  X[  and  Xj  were  the  two  of  our  three  targets 
that  produced  a  measurement,  then  we  could  write  the  solution  for 
P(Xi&Xj|Z,2)  as 


P(X*  &X- 12  2)  =  *P(Xi&Xj  generate'Z|2) 

'  '  [p(t|  X]  &^2 .2)  generateZ|2) 

^  *2)  P(X2&X3  generateZ|2) 

+  P(  Z|  X1&X3 ,2)  P(Xi&X3  generate  Z 1 2)  j 


(41) 


distribution  of  Z(k)  about  the  two  tracks  that  generated  it,  and 
P(Xj&Xj  generate  Z|2)  is  the  probability  that  the  ith  and  jth  targets 
combine  to  generate  a  measurement.  Now  we  have  to  make  some  as¬ 
sumptions. 


We  assume  that  Z(k)  is  Gaussianly  distributed  somehow  around  the 
two  targets  that  generated  it.  Let  us  assume  that  the  distribution’s  mean  is 

the  midpoint  between  Xi  and  Xj  as  shown  in  Figure  10.  Then  the  den¬ 
sity  function  we’re  looking  for  is 


P(z|jq&‘^,2)  =  [<2ir)'^|Sdij|]' 

— ^  S  T?  "v  ... 

•  e  2  "^dij  dij  ^dij 


where  M  is  the  dimension  of  Z(k)  and 


(42) 


ESTIMATED 

RANGE 


APPROXtMATE  P^fXi)  FROM 
ESTIMATED  RANGE  OF  TARGET 


Pd(Xj) 


Sj  &  x^i  from 
ASS./CORR.  ^ 

CALCULATE  pdf  VALUE 

p{Z|Xi.i)-  [{27r)M  .iSii]‘0-5  •  e-os»x^i 

P(z|Xj.i: 


CALCULATE  p(Z|Xnt.  1)  •  Pd(Xnt) 
APRIORI  AND  STORE  IN  MEMORY 

P(Z|Xnt.  1)  =  [{2'rr)*''*  •|5Rl]"°‘® 

Pd  (^nt)  =  CONSTANT 

p{Z|Xnt.  1)  •  Pd(Xnt) 


CALCULATE  PfXj  |Z) 


P(XriZ)  = 


P(2|X|.1)  •Pd(Xi) 


P(Xi|2) 


2  [p(Z  IXn.D  •  Pd(Xn)]  +  P(Z|Xnt.1)  •  P^(X„t) 
n=l 


Figure  9.  ESM/Radar  Probability  Calculation 
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Sdij  [HiPiHiT  +  HjPjHjT  +  HjPyHjT  +  R  (44) 

Note  that  Hj  is  the  transformation  of  Xj  to  measurement-space,  Pj  is 
the  error  covariance  matrix  for  Xi  .  Pij  =  Pji’^  is  the  cross-error  covari- 
ance  matrix  between  Xj  an<3  Xj  -  and  R  is  the  measurement  noise  co- 
variance.  Next  we  must  find  P(Xi&Xj  generate  Z|2). 

The  probability  of  two  and  only  two  targets  generating  a  measurement 
depends  on  the  probability  of  those  two  targets  contributing  to  any  meas¬ 
urement.  Assuming  at  most  three  targets  will  contribute  to  a  single  meas¬ 
urement  gjves  (from  the  Venn  Diagram.  Figure  8) 

P(Xi&Xj  generate  Z)  =  P(Xi&Xj  will  contribute)  -  P(3)  (45) 

where  P(Xi  will  contribute)  is  the  probability  that  Xj  and  Xj  con¬ 
tribute  to  a  measurement  and  P(3)  is  the  probability  that  all  three  com¬ 
bine  to  generate  the  measurement.  We  find  the  conditional  probability  by 
conditioning  ?(Xi  &Xj  generate  Z)  with  the  event  that  two  and  only  two 

targets  generate  Z(k). 


From  the  Venn  Diagram 


&Xj  generate  Z|2)  = 


P(Xi  &Xj  will  contribute)  -  P(3) 


P(l)  is  the  probability  that  one  and  only  one  target  generated  Z(k) 
and  P(2)  is  the  probability  that  two  and  only  two  generated  Z(k).  If  we 
knew  P(Xi&3ij  will  contribute),  then  we  could  find  P(l).  P(2),  and 
P(3).  P(l)  is  the  probability  that  neither  two  nor  three  targets  combined 
to  generate  a  measurement. 

p(l)  =  [l-P(Xi&X2  will  contribute)] 

•  [l-P(X2&X3  will  contribute)]  (47) 

•  [l-P(Xi&X3  will  contribute)]. 


P(2)  =  P(Xi&X2  contribute) -I-  P(X2&X3  will  contribute) 
+  P(Xi&X3  will  contribute)  -  3P(3). 


and  because  events  Q)'  ®  make  up  a  mutually  exclusive  set 

P(3)  =  1  -  P(l)  -  P(2)  or 


P(3)  =  [p(l)  +  P(Xi  &X2  contribute) 

-I-  P(X2&X3  will  contribute) 

+  P(  X^  &X3  will  contribute)  -  l] . 


We  must  focus  our  efforts  on  finding  P(Xi&Xj  will  contribute). 

Let  us  assume  that  whenever  the  distance  betweenj^argets  i  and  j  in 
the  measurement -space  is  less  than  some  threshold.  ^7.  then  they  will 
definitely  be  ambiguous  and  they  will  contribute  to  a  single  measurement 
(if  they  are  within  detection  range) .  Then  if  we  define 

^  Ck  ^  -fc 

AXij  =HiXi  -HjXj  and  AXij  =  Hj  Xj  -  Hj  Xj . 


P(Xi  &Xj  will  contribute)  ^  ^ 

=  p(-a<  AXjj  <  aldet)*  Pcj  (Xj &Xj) 

where  P(-a<  A'^j  <  oldet)  is  the  probability  that  given  Xj&Xj  are 
detectable,  they  combine  and  generate  a  single  measurement  because 
AXjj  is  less  than  O,  and  P^CXj  &  Xj)  is  the  probability  of  detectii^  the 
combined  Xj  &  Xj  target.  We  reason  that  the  actual  value  for  AXjj  is 

Gaussianly  distributed  about  A  Xij .  then 

AXjj  -  AXjj )  s^x..  ( AXjj ) 

•  e  ‘j 
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=  HiPjHiT  +HjPjHjT_HjPj.HjT  _  HjPjiH/. 


(52) 


we  will  have  to  estimate  based  on  target  range  estimates,  and 
P(-0  <  AXjj  <  o  |det)  is  the  integral  of  the  density  function 

P(-a<  A'^j  <  oldet)  = 

-cyj^-Oi-Gi  d(Axpd(AX2).r.d(AXj^) 

which  would  require  a  multidimensional  numerical  integration.  If  the  off- 
diagonal  terms  in  are  relatively  small  then  we  could  say  that  the 

components  AXj,  AX2,  AXj^  are  uncorrelated  and  write 


P(-a<  AXij  <  aldet)  =  ^(Ax^,)  ... 

where 


^AXn^^^n)  “ 


_i_ 


V27TS  > 


1 


.e''(AXn-AXn)^/(SAXnn> 


(55) 


is  much  more  tedious.  The  probability  P^X^  /  Z)  when  Xj  is  one  of  the 
three  closest  targets  to  Z(k)  is  (for  discussion  let  i=l) 


r{^|z)  =  P(3q|z.i)  •  p(i)  +  [p(3q  &X2IZ.2) 

+  ?i%&%\z,2)]*  P(2)  +  P(3) 


(56) 


where  we  have  summarized  the  necessary  calculations  in  Figure  1 1 .  The 
calculations  are  extensive  and  their  complexity  probably  outweighs  the 
benefits  which  probabilistic  data  association  brings  to  the  Multiple  Target/ 
Raid  Cell  Assessment  problem. 


GJVEN  THAT  X|  &  Xj  DEFINITELY  GENERATE  A 
MEASUREMENT.  THAT  MEASUREMENT'S  LOCATION 
IS  GAUSSIANLY  DISTRIBUTED  AROUND  THE 
MIDPOINT  OF  (X|  ,Xj) 


Figure  JL  Ambiguous  Target  Solution 

5.0  CONCLUSIONS 

When  applied  to  situations  where  each  measurement  is  uniquely  asso¬ 
ciated  to  a  single  source,  such  as  the  ESM/Radar  problem,  probabilistic 
data  association  offers  the  cautious  approach  required  for  fusing  ID  data 
to  kinematic  tracks.  But  when  the  sources  may  be  ambiguous  and  the 
measurement  is  not  unique,  the  effort  required  to  calculate  P(Xj  /"z)  is 
too  extensive  to  perform  in  real-time.  Thus,  each  application  requires  a 
thorough  analysis  before  applying  probabilistic  data  association  tech¬ 
niques. 


4.1  Ambiguous  Track/Raid  Cell  Algorithm  Summary 

Tl^  completes  the  derivation  of  the  terms  we  require  to  compute 
P(Xi  /Z)  in  the  ambiguous  target  case.  In  summary,  the  modifications  we 
must  make  to  the  Association/Correlation,  the  Kinematic  Fusion,  and  the 
Attribute  Fusion  functions  are  the  same  as  the  ones  we  make  in  the 
ESM/Radar  problem.  The  difference  is  that  the  calculation  of  P(Xj/'z) 
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This  paper  develops  a  group  tracking  algorithm  for  the  tracking  of  clusters  of  closely 
spaced  targets  as  viewed  by  passive  sensors.  This  problem  is  complex  because  the  size  and  shape 
of  the  observed  cluster  will  differ  from  sensor  to  sensor.  Since  a  passive  sensor  provides  a 
projection  of  the  objects,  passive  sensors  in  different  locations  will  provide  different 
projections  of  the  3-dimensional  (3-D)  shape  of  a  cluster.  Furthermore,  the  number  of  resolved 
objects  in  a  cluster  can  vary  as  closely  spaced  targets  may  appear  to  a  sensor  as  a  single 
extended  object. 

In  order  to  track  the  target  clusters  with  multiple  sensors,  a  method  is  needed  to 
characterize  the  size,  shape  and  location  of  each  cluster.  The  method  described  in  this  paper 
models  the  target  cluster  in  3-D  as  an  ellipsoid  with  the  projection  in  2-D  being  an  ellipse. 

The  centroid  and  parameters  of  the  ellipsoid  are  tracked  over  time  using  measurements  from 
widely  spaced  passive  sensors.  The  filtering  methods  are  described  and  performance  is  presented 
based  upon  tracking  simulations.  Application  of  this  method  to  tracking  extended  objects  with 
multiple  sensors  is  also  discussed. 
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INTRODUCTION 


This  paper  presents  a  group  tracking  * 
method  that  has  been  developed  for 
surveillance,  such  as  for  the  midcourse 
stage  of  the  Strategic  Defense  Initiative 
(SDI) .  A  cluster  of  targets,  consisting  of 
one  reentry  vehicle  (RV)  and  40  decoys,  is 
assumed  to  be  deployed  from  a  post  boost 
vehicle  (bus).  Initially,  the  members  of 
the  cluster  will  have  small  random 
velocities  with  respect  to  the  center  of 
mass  of  the  cluster.  However,  as  the 
cluster  separates  the  members  of  the 
cluster  will  experience  different 
gravitational  forces  due  to  their  different 
positions  within  the  earth’s  gravitational 
force  field.  This  phenomenon,  denoted  the 
gravity  gradient  effect,  will  tend  to 
further  increase  the  relative  velocities 
and  to  spread  the  cluster  further  apart 
along  the  direction  of  the  gravitational 
force. 

As  addressed  in  this  paper  the 
tracking  process  will  be  performed  using 
observations  from  passive  Electro  Optical 
(EO)  sensors.  The  EO  sensors  measure  the 
angles  from  the  platform  to  the  target. 

Then,  assuming  the  platforms  are  typically 
separated  by  large  distances,  it  is 
possible  to  determine  the  three  components 
of  a  target's  position  by  the  process  of 
triangulation  (stereo  tracking) .  However, 
as  discussed  below,  it  will  not  be  possible 
to  initially  track  the  members  of  the 
closely  spaced  target  cluster  as 
individuals . 

In  general,  it  is  both  inaccurate  and 
wasteful  of  computer  resources  to  attempt 
to  form  individual  tracks  on  closely  spaced 
targets  that  have  essentially  the  same 
velocity  [1].  This  is  particularly  true 
if,  as  in  this  case,  the  individual  targets 
may  not  be  resolvable.  Note  that  for  EO 
sensors  there  are  only  two  dimensions  for 
resolution  and  that,  since  different  sensor 
systems  will  have  different  viewing  angles, 
the  particular  set  of  targets  that  is 
resolvable  will  vary  with  the  sensor. 

Thus,  for  the  purposes  of  track  stability 
and  efficient  resource  allocation,  group 
tracking  is  desirable. 


Due  to  sensor  measurement  resolution 
limits,  closely  spaced  targets  may  appear 
to  the  sensor  as  a  single  extended  object. 
Typical  resolution  history  is  illustrated 
by  Figure  1,  which  shows  the  number  of 
resolved  targets  plus  clumps  observed  by  a 
sensor  as  a  function  of  time  from 
deployment.  A  clump  is  defined  to  be  an 
extended  object  containing  at  least  two 
unresolved  targets.  The  total  number  of 
resolved  targets  plus  clumps  is  the  number 
of  elements  in  a  measurement  of  a  group  (or 
cluster)  of  targets.  Results  given  in 
Figure  1  are  for  a  cluster  of  40  targets 
and  are  shown  for  two  spherically  uniform 
distributions  of  the  random  velocity  with 
respect  to  the  centroid.  The  limits  on  the 
magnitude  of  the  velocity  are  0.5  and  2.0 
meters/sec  for  the  upper  curve  and  0  and 
2.0  meters/sec  for  the  lower  curve. 

In  general,  it  may  be  possible  to 
estimate  the  number  of  individual  targets 
contained  in  an  extended  object  based  upon 
the  shape  and/or  brightness  of  the  extended 
object.  However,  for  the  purpose  of  this 
study,  we  considered  the  worst  case  where 
an  extended  object  was  indistinguishable 
from  an  individual  target.  Thus,  the 
problem  is  to  develop  a  tracking  method  for 
a  cluster  (or  group)  of  objects  in  which 
some  closely  spaced  elements  may  not  be 
resolvable  by  the  sensors  at  certain 
viewing  angles.  The  term  cluster  will  be 
used  to  refer  to  a  collection  of  objects 
with  similar  state  vectors.  The  term  group 
will  be  used  to  refer  to  an  apparent 
cluster  of  objects  based  on  sensor 
measurements . 

The  open  literature  was  searched  for  a 
group  tracking  approach  based  on  multiple, 
passive  sensors.  Of  particular  interest  is 
an  approach  that  is  applicable  to 
birth-to-death  tracking  and  central-level 
processing  [3].  While  there  are  a  number 
of  group  tracking  approaches,  none  appear 
to  be  suitable  for  this  task. 

In  developing  the  new  approach,  a 
number  of  critical  issues  had  to  be 
addressed,  as  follows.  How  to  represent 
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the  cluster  in  3-dimensions  (3-D)  and  in 
2-dimensions  (2-D)  so  that  the  3-D 
parameters  can  be  estimated  from  2-D 
parameters?  How  to  track  the  3-D 
parameters  over  time  and  how  to  compute  the 
2-D  parameters  from  the  passive  sensor 
measurements?  And  finally/  how  to 
accommodate  the  sensor-to-sensor 
differences  in  the  apparent  size,  shape  and 
composition  of  the  group  and  the  time 
varying  characteristics  of  the  cluster? 
The  following  sections  describe  the  concept 
that  was  developed  and  results  of  the 
feasibility  simulation  that  was  conducted. 


THE  GROUP  TRACKING  APPROACH 

The  group  tracking  concept  that  has 
been  developed  characterizes  the  target 
cluster  by  its  centroid  and  extent  about 
the  centroid.  The  extent  is  represented  in 
three  dimensions  by  an  ellipsoid  which 
becomes  an  ellipse  when  viewed  in  the  two 
dimensional  focal  plane  of  an  EO  sensor 
(Figure  2).  As  a  consequence,  it  was 
possible  to  develop  a  method  to  track  the 
parameters  of  the  ellipsoid  using  multiple 
separated  EO  sensors. 

The  group  centroid  can  be  tracked  with 
respect  to  the  Cartesian  Earth  Centered 
Inertial  (ECI)  coordinate  system.  This 
estimated  centroid  defines  the  center  of 
the  ellipsoid  enclosing  the  group.  The 
extent  of  the  ellipsoid  is  defined  by  the 
displacement  vectors 

6x'^  =  [6x,6y,6z] 

6x,6y,6z  =  displacements  from  ellipsoid 
center  such  that 


Then,  the  angular  displacements  from  the 
centroid  are  related  to  the  ECI  coordinate 
displacements  through 


where 


=  E[6n^]/  =  E[6n*6e] 

n  HE 

The  approach  is  to  form  a  group 
measurement  each  time  a  new  set  of ^ 
observations  is  received.  First,  in  order 
to  form  a  measurement  gate  about  the 
predicted  centroid,  an  estimate,  P  ,  of  the 
ellipse  parameters  can  be  obtained  using 
Eg.  3  and  the  estimated  ellipsoid 
parameters.  Then,  the  measurement 
distribution  is  defined  by  the  ellipse 
parameters  and  the  sensor  measurement  error 
variances.  Assuming  the  contribution  from 
the  measurement  error  to  be  small  relative 
to  the  dispersion,  a  measurement  with 
angular  displacements  {Sr\,Sz)  from  the 
predicted  centroid  will  satisfy  the  gate  if 

<Sn,6e)  (*2)  <  G  (4) 

The  gating  constant,  G,  in  Eg.  4  was  chosen 
to  be  10  for  the  results  to  be  presented 
below. 
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Complex,  but  readily  obtained, 
expressions  can  be  used  to  relate  the 
angles  (n,e)  measured  by  the  EO  sensors 
to  the  target  position  (x,y,z)  in  the  ECI 
coordinate  system. 


Thus,  to  summarize,  all  observations 
satisfying  Eg.  4  will  be  considered  for  use 
in  updating  the  group  track  in  question. 

In  general,  an  observation  may  satisfy 
multiple  group  track  gates  so  that  a 
conflict  resolution  logic  is  required. 
However,  for  this  preliminary  study  we  have 
only  considered  a  single  group  track. 
Finally,  all  observations  that  are  assigned 
to  the  group  track,  by  virtue  of  satisfying 
Eg.  4  for  this  case,  are  then  used  to  form 
a  group  observation  consisting  of  a 
measured  centroid  and  dispersion  ellipse. 
The  measured  centroid  and  dispersion 
parameters  are  defined  in  the' standard 
manner,  for  N  individual  observations 
included  in  the  group  observation. 
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n  -  fj^(x,y,z)  ,  E  =  f2(x,y,z) 


with  similar  expressions  for 
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The  measured  quantities  are  used  to 
update  the  group  track  estimates  using  the 
Kalman  filtering  techniques  described  in 
the  next  section.  The  entire  process  is 
summarized  by  the  flow  chart  shown  in 
Figure  3. 
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KALMAN  FILTERING  METHODS 

Separate  Kalman  filters  can  be  used 
for  estimating  the  target  centroid  and 
ellipsoid  extent  parameters.  Both  filters, 
outlined  below,  use  standard  extended 
Kalman  filtering  methods. 

Centroid  Filter 

The  centroid  filter  is  a  six  state 
filter  using  position  and  velocity  for  the 
three  Cartesian  components  in  the  EC^ 
coordinate  system.  Extrapolation  was  done 
in  ten  second  intervals.  Extrapolation 
over  this  interval  is  performed  using  a 
trajectory  in  which  the  gravitational  force 
at  the  center  of  the  extrapolation  period 
is  computed  and  then  used  as  a  constant 
throughout  the  interval.  Results  obtained 
using  this  approximate  extrapolation  method 
have  been  found  to  closely  correspond  to 
those  derived  using  more  exact  methods. 

The  initial  position  and  velocity 
estimates  are  taken  to  be  the  same  as  those 
for  the  bus  which  deploys  the  target 
cluster.  It  is  assumed  that  the  bus  is 
being  tracked  and  the  initial  covariance 
matrix  elements  are  chosen  using  the 
expected  bus  tracking  accuracies. 

The  elements  of  the  measurement  vector 
are  the  centroid  azimuth  and  elevation 
angles  as  computed  according  to  Eq.  5.  The 
measurement  vector  is  related  to  the  state 
estimation  vector  in  the  standard  manner. 

y  =  Hx  +  V  (6) 

where  H  is  defined  in  Eq.  2,  x  is  the  state 
vector  and  y  is  measurement  noise.  The 
measurement  noise  covariance  matrix  is 
taken  to  be  a  function  of  the  actual 
dispersion  and  the  number  of  elements  in 
the  group  measurement.  The  centroid 
measurement  noise  is  assumed  to  decrease  as 
more  elements  of  the  group  are  detected. 

The  process  noise  covariance  matrix,  Q,  was 
taken  to  have  the  form  associated  with  the 
discretized,  continuous-time  white  process 
noise  [2]  and  with  the  magnitude  of  the 
acceleration  variance  chosen  to  compensate 
for  the  effects  of  the  gravity  gradient. 

Extent  Parameter  Filter 

The  extent  parameter  filter  also  uses 
six  states.  These  are  the  parameters  given 
in  Eq.  1  that  define  the  ellipsoid.  Using 
Eq .  3  it  is  straightforward  to  find  a 
linear  relationship  between  the  measurement 
vector  y  and  the  state  estimation  vector  x 

X  =  X  +  X  (7) 


The  elements  of  the  target  cluster 
were  deployed  from  the  bus  with  random 
velocities  relative  to  the  velocity  of  the 
centroid.  Therefore,  initially,  before  the 
effects  of  the  gravity  gradient  become 
appreciable,  the  cluster  extent  grows 
linearly  with  time.  This  means  that  the 
extent  parameters  grow  with  the  square  of 
time  since  deployment.  For  this  study  it 
was  assumed  that  an  accurate  estimate  of 
the  time  of  deployment  could  be  obtained. 
Thus,  the  transition  matrix  for  the  process 
from  time  step  k  to  k+1  was  taken  to  be 

4>(k+l,k)  =  a^I 

where  I  is  the  six  by  six  identify  matrix 
and 


As  the  cluster  spreads  apart,  the 
effect  of  the  gravity  gradient  is  to 
further  increase  the  dispersion  rate  along 
the  direction  of  the  gravitational  force. 

An  approximate  model  was  derived  and  used 
to  aid  the  filter  through  increasing  the 
process  noise  covariance  matrix  to  account 
for  the  perturbations  due  to  the  gravity 
gradient.  A  more  exact  method  in  which  the 
actual  dispersion  rate  is  estimated  taking 
into  account  the  known  effect  of  the 
gravity  gradient  may  improve  performance. 
However,  as  will  be  illustrated  by  results 
to  follow,  the  artifice  of  making  the 
process  noise  covariance  matrix  a  function 
of  the  gravity  gradient  leads  to  an 
adequate  response  of  the  filter  to  this 
effect . 

As  discussed  in  [1]  the  measurement 
noise  covariance  matrix  was  made  a  function 
of  the  expected  dispersion  and  the  number 
of  elements  in  the  group  measurement.  As 
with  the  centroid  measurement,  the  extent 
parameter  measurement  accuracy  is  assumed 
to  increase  with  the  number  of  elements  in 
the  group  measurement.  Finally,  the 
initial  state  estimates  and  covariance 
matrix  were  defined  based  upon  expected 
random  deployment  velocity  statistics. 


THE  EXTENDED  OBJECT  TRACKING  APPROACH 

An  extended  object- can  also  be  tracked 
with  the  multiple  passive  sensors  using 
this  same  approach.  There  are  a  number  of 
methods  that  can  be  used  to  establish  the 
parameters  of  an  ellipse  and  the  centroid 
for  an  extended  object  as  observed  by  a 
single  passive  sensor.  The  extended  object 
can  be  represented  by  an  ellipsoid  in  3-D 
just  as  was  described  for  a  cluster. 


where 
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and  Hn^  is  a  three  by  six  matrix  whose 
elements  are  found  using  Eq.  3.  Again,  y 
represents  the  associated  measurement  noise. 


Using  multiple  sensors,  Eq .  7  can  be 
used  to  estimate  the  parameters  of  the 
ellipsoid.  The  centroid  in  3-D  can  be 
estimated  using  Eq.  6.  Thus,  Kalman 
filters  can  be  used  to  track  the  3-D 
parameters.  However,  if  the  actual  size  in 
3-D  of  an  object  is  varying,  then  the 
dynamic  equation  used  in  the  Kalman  filter 
must  account  for  that  change. 
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SIMULATION  STUDY 

A  simulation  was  developed  to  show 
feasibility  of  the  group  tracking  method 
defined  above.  This  simulation  considered 
a  typical  30  min  (1800  sec)  scenario  with  a 
single,  isolated  cluster  as  described  below. 

Scenario  Description 


varied.  The  first  results,  given  in  Figure 
5,  show  the  total  distance  from  the 
centroid  estimate  to  the  true  target  group 
centroid.  For  the  case  of  infinite 
resolution,  the  centroid  estimation  error 
is  essentially  zero  but  when  resolution 
effects  are  considered  the  distance  is  in 
the  order  of  100  to  200  meters  throughout 
the  run. 


For  this  study,  two  sensor  platforms 
were  observing  the  dispersion  of  40  target 
objects  which  were  deployed  from  a  target 
bus  at  an  altitude  of  500  km.  It  was 
assumed  that  the  bus  vjas  previously  tracked 
as  an  individual  target  and  that  group 
tracking  of  the  40  objects  was  initiated  at 
this  altitude.  The  target  objects  were 
each  given  an  initial  relative  velocity  to 
simulate  ejection.  The  relative  velocities 
were  imparted  in  a  random  direction  with 
magnitudes  uniformly  distributed  between  0 
and  2  m/s.  The  target  objects  were  on  a 
ballistic  trajectory.  The  sensor  platforms 
were  in  circular  orbits  with  altitudes  of 
3000  km.  One  orbit  was  inclined  40®,  the 
other  80®. 

The  angular  resolution  of  the  sensors 
was  taken  to  be  0.001  degrees  in  azimuth 
and  elevation.  However,  to  simplify  this 
preliminary  study  no  measurement  errors 
were  introduced. 

Coordinate  System  Definition 

The  position  of  the  target  in  the 
sensor  f ield-of-view  (FOV)  was  defined  by 
two  angles,  azimuth  and  elevation.  These 
angles  are  measured  in  the  Spacecraft 
Center  True  of  Date  (SCT)  coordinate  system 
which  is  centered  at  the  current  spacecraft 
location.  The  fundamental  plane  of  the  SCT 
system  is  the  plane  of  the  orbit  and  the 
fundamental  axis  points  toward  nadir. 
Azimuth  and  elevation  of  the  target  are 
then  defined  in  this  coordinate  system  as 
shown  in  Figure  4. 

Presentation  of  Results 


Results  will  be  presented  for  the  case 
where  the  two  sensors  observe  the  target 
cluster  every  100  sec.  Results  derived  for 
shorter  (10,50  sec)  sampling  intervals  were 
somewhat  better  but  the  performance 
improvement  did  not  seem  to  compensate  for 
the  increased  computational  requirements 
needed  to  process  the  additional  data. 

Two  resolution  conditions  were 
examined.  For  the  first  (infinite 
resolution)  it  was  assumed  that  all  targets 
were  observed  each  look.  For  the  second 
case  (finite  resolution)  all  targets  within 
0.001  degrees  were  assumed  to  be 
unresolvable.  This  led  to  the  formation  of 
extended  objects  which  were  clumps  of 
unresolved  closely  spaced  targets.  For 
this  study  extended  objects  were  used  by 
the  measurement  process  as  though  they  were 
indistinguishable  from  single  targets. 
typical  resolution  history  for  this  case  is 
shown  by  the  lower  curve  in  Figure  1. 

Results  are  shown  for  typical  runs. 

No  Monte  Carlo  statistics  were  compiled. 
However,  the  same  type  of  results  have  been 
obtained  for  a  number  of  individual  runs  in 
which  the  random  number  sequences  have  been 


We  next  consider  performance  of  the 
extent  estimation  process.  First,  Figure  6 
shows  the  ratio  of  the  estimated  to  the 
true  volume  of  the  ellipsoid  defining  the 
target  group  extent.  The  volume  of  the 
ellipsoid  defined  by  Eq.  1  is  given  by 


V  =  J\^  Td 

3  v  '  xyz ' 


Thus,  the  volume  ratio 


is  defined  as 


where  P  in  computed  using  the  true 
target  positions  and  is  computed  using 

the  state  vector  estimates. 

Referring  to  Figure  6,  note  that  Ry 
is  close  to  unity  throughout  the  run  when 
all  targets  are  resolved.  However,  this 
ratio  is  considerably  larger  than  unity 
during  the  early  stages  of  the  run  when 
finite  resolution  is  considered.  This  is 
because  the  objects  with  smaller  dispersion 
velocities  tend  to  be  those  that  remain 
closely  spaced  and  are  thus  seen  as 
extended  objects  rather  than  individuals. 
This  tends  to  distort  the  apparent 
ellipsoid  by  putting  less  weight  on  those 
group  members  that  are  close  to  the 
centroid.  Thus,  the  apparent  ellipsoid 
dispersion  parameters  are  computed  to  be 
too  large.  This  problem  could  be  reduced 
if  extended  objects  were  recognized  as 
consisting  of  more  than  one  object  and 
weighted  accordingly. 

In  addition  to  considering  the  total 
volume  it  is  important  to  compare  the  shape 
and  orientation  of  the  estimated  with 
respect  to  the  true  ellipsoid.  This 
comparison  was  performed  by  computing  the 
eigenvalues  of  the  estimated  dispension 
matrix  relative  to  the  true  dispension 
matrix.  This  is  achieved  by  solving 
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for  all  three  solutions  for  the  eigenvalue, 
X.  This  is  equivalent  to  computing  the 
length  along  the  principle  axes  of  the 
estimated  ellipsoid  given  that  the  true 
ellipsoid  is  a  sphere.  Thus  the 
eigenvalues  can  be  viewed  as  the  extreme 
dispension  ratios  and  the  ideal  values  are 
unity.  Figures  7  and  8  show  these 
results.  Again,  the  accuracy  of  the 
estimation  process  when  all  targets  are 
resolved  is  excellent  but  the  distortions 
associated  with  the  finite  resolution  case 
are  apparent. 
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Probably  the  most  important  test  of 
the  estimation  process  is  the  success  with 
which  the  estimates  of  the  ellipsoid  and 
the  projected  ellipses  actually  contain  the 
target  group.  First#  the  theoretical 
probability  of  a  true  group  member  falling 
outside  the  ellipsoid  defined  by  Eg.  1  can 
be  computed  as  a  function  of  D  using 
chi-square  statistics  with  three  degrees  of 
freedom.  For  example#  choosing  D  =  8.0  and 
13.0  gives  probabilities  of  0,05  and  0.005# 
respectively,  of  a  true  target  position 
falling  outside  the  volume  defined  by 
Eg.  1.  Results  indicated  that  throughout 
the  duration  of  the  run  and  for  both  finite 
and  infinite  resolution  there  were 
essentially  no  targets  outside  the  volume 
defined  by  D=13.  Also#  an  average  of  about 
1.5  targets  per  40  target  cluster  were 
outside  the  volume  defined  by  D=8.0.  These 
results  agree  well  with  the  theoretical 
predictions . 

The  gate  defined  by  Eg.  4  with  gating 
constant  G=10  would  be  expected  to  contain 
a  true  target  return  with  probability  of 
about  0,995.  This  is  computed  from  chi- 
square  variable  statistics  with  two  degrees 
of  freedom.  In  practice#  the  simulation 
results  showed  that  there  were  no  returns 
outside  the  gate  for  either  the  finite  or 
the  infinite  resolution  cases.  This  was 
true  for  each  of  the  18  observations 
(received  at  100  sec  intervals  over  the 
1800  sec  tracking  period) . 

Figures  9  through  12  illustrate  gating 
behavior  by  showing  the  gates  and  the 
targets  positions  as  projected  onto  the 
field  of  view  of  one  sensor.  Results  are 
shown  at  400  and  1600  sec.  and  for  the 
finite  and  infinite  resolution  cases.  It 
can  be  noted  that  the  shape  of  the  cluster 
changes  with  time  due  to  the  effects  of  the 
gravity  gradient  but  that  the  shape  of  the 
gate  also  changes  accordingly.  Note  the 
dispersion  is  reflected  in  the  change  of 
scale  between  Figures  9  and  10  and  Figures 
11  and  12. 

Referring  to  Figures  10  and  12#  the 
symbols  A  and  □  refer  to  extended  objects# 
as  formed  by  the  finite  resolution  case# 
consisting  of  two  and  more  than  two 
targets#  respectively.  The  effect  of 
unresolved  targets  is  seen  to  be  an 
increase  in  the  gate  size#  but  for  all 
cases  the  gates  correspond  well  to  the  true 
target  positions. 

The  results  presented  above  used  a  uniform 
initial  velocity  distribution  with  maximum 
relative  velocity  of  2m/s.  Other  runs  were 
made  in  which  the  maximum  velocity  was 
increased  to  5m/s  and  the  method  performed 
equally  well . 


is  based  upon  the  volume  and  shape  of  the 
estimated  ellipsoid  and  the  high  proportion 
of  targets  enclosed  by  the  estimated 
ellipsoid.  Also,  gating  tests  based  upon 
the  elliptical  projection  of  the  ellipsoid 
onto  the  sensor  field  of  view  have  been 
shown  to  include  the  true  target  returns. 

Much  future  study  is  required  to 
further  define  the  method.  Logic  for  the 
transition  from  group  to  individual  target 
tracking  have  yet  to  be  tested.  Also,  the 
concepts  that  have  been  identified  to 
handle  crossing  groups  and  merging  groups 
must  be  developed  and  tested.  Corrections 
to  the  measurement  vector  must  be  added  to 
adjust  for  the  bias  caused  by  measurement 
errors.  Finally,  it  may  be  desirable  to 
more  accurately  represent  the  effects  of 
the  gravity  gradient  that  becomes  an 
important  factor  in  group  dispersion  as  the 
targets  separate. 
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CONCLUSION/FUTURE  STUDY 

This  paper  has  summarized  the 
preliminary  development  and  evaluation  of  a 
new  group  tracking  algorithm  for  use  in 
distributed#  passive  sensor  applications. 

It  has  demonstrated  the  feasibility  of  this 
approach  for  group  tracking.  In 
particular,  a  close  match  between  the 
estimated  ellipsoid  taken  to  represent  the 
target  cluster  and  the  true  target 
positions  has  been  found.  This  conclusion 
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Figure  7.  Infinite  Resolution  Dispersion  EstimatejEigervalues 


Figure  9.  Gating  at  400  sec  for  Infinite  Resolution  Case 
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Figure  8.  Finite  Resolution  Dispersion  Estimate  Eigervalues 


Figure  10.  Gating  at  400  sec  for  Finite  Resolution  Case 


Figure  1 1 .  Gating  at  1600  sector  Infinite  Resolution  Case 


Figure  1 2.  Gating  at  1 600  sec  for  Finite  Resolution  Case 
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Figure  1 .  Resolution  History  for  a  40  Target  Cluster 


Figure  2.  A  Target  Cluster  as  seen  by  Two  Passive  EO  Sensors 
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Figure  3.  Group  Tracking  Flow  Chart 


Figure  4.  Sensor  Measurement  Geometry 
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Figure  5.  Centroid  Position  Estimation  Error 


Figure  6.  Ellipsoid  Volume  Ratio 
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Data  fusion  will  establish  synergism  amongst  sensors  thereby  extending  the  combatant's  sit¬ 
uational  awareness.  When  the  fusion  concept  is  applied  to  multiple  combatants  in  cooperative 
and  coordinated  tactics,  it  will  suffer  for  want  of  communications. 

Simply  depicted,  a  combat  platform  consists  of  three  basic  systems;  a  weapons  system,  sen¬ 
sor  system  and  command  and  control  system.  On  a  single  platform,  the  unification  of  these  ele¬ 
ments  enjoys  advantages  that  are  not  available  in  a  raultiplatf orm  cooperative  effort. 

What  happens  when  the  sensors,  weapons  and  command  and  control  are  not  collocated  in  a  sin¬ 
gle  platform? 

The  difficulty  of  correlating  track  reports  within  a  Battle  Force  is  well  known.  Two  or 
three  platforms  perceiving  the  same  target  are  likely  to  report  two  or  three  separate  track  pos¬ 
itions  due  to  several  contributing  factors. 

Compensation  for  these  errors  can  be  established  and  eventually  data  fusion  developed  be¬ 
tween  platforms,  but  it  requires  extensive  communications  which  will  not  be  available  in  a  hos¬ 
tile  combat  environment.  The  present  state  of  communications  development,  as  is  relates  to  com¬ 
bat,  is  driven  by  a  simplified  threat  which  ignores  the  multifacets  of  communications  in  elec¬ 
tronic  warfare.  This  paper  addresses  what  needs  to  be  specified  in  tactical  combat  communica¬ 
tions  for  supporting  coordinated  tactics  in  an  electronic  warfare  environment. 
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INTRODUCTION 

There  has  been  considerable  progress  in 
the  development  of  software  systems  support¬ 
ing  tactical  "real  time"  decision  making. 
Applications  of  these  developments  have  pri¬ 
marily  addressed  the  tactical  decisions  con¬ 
tained  within  a  single  platform.  In  the 
April  1988  issue  of  the  Naval  Institute  Pro¬ 
ceedings,  CDR  Kendall  King  described  such  a 
systems  application  for  a  ship  board  support 
system  as  shown  in  Figure  1.  As  a  conclud¬ 
ing  sentence  to  this  section  of  his  article, 
he  stated,  "Capabilities  could  be  expanded 
even  more  by  communicating  with  tactical 
management  information  systems  on  other 
ships  or  at  other  activities  via  data 
links".  The  RDT&E  thrust  is  now  turning  to 
expansion  of  cooperative  or  coordinated  tac¬ 
tics,  tying  together  many  platforms  and  de¬ 
veloping  the  synergistic  effect  of  multiple 
sensors  and  weapons. 

Simply  depicted,  a  combat  platform  con¬ 
sists  of  three  basic  systems;  a  weapons  sys¬ 
tem,  sensor  system  and  command  and  control 
system.  On  a  single  platform,  the  unifica¬ 
tion  of  these  elements  enjoys  several  advan¬ 
tages  that  are  not  available  in  a  multiplat¬ 
form  cooperative  effort.  Several  of  the  ad¬ 
vantages  are:  (1)  the  bore  sighting  of  sen¬ 
sors  in  relation  to  each  other,  (2)  the  di¬ 
rect  alignment  of  the  weapons  to  a  host  sen¬ 
sor  integrator,  (3)  grid  lock  of  targets  and 
weapons  to  a  single  navigation  reference, 
and  (4)  an  internal  communications  capabil¬ 
ity  to  transfer  massive  amounts  of  informa¬ 
tion  without  concern  of  propagation  con¬ 
straints  or  intentional  interference. 

What  happens  when  the  sensors,  weapons 
and  command  and  control  are  not  collocated 
in  a  single  platform? 

The  difficulty  of  correlating  track  re¬ 
ports  within  a  Battle  Force  is  well  known. 
Two  or  three  platforms  perceiving  the  same 
target  are  likely  to  report  two  or  three 
separate  track  positions  due  to  several 
contributing  factors.  For  example:  (1)  the 
radar  misalignment ' to  the  bore  sight  of  an 
aircraft,  compensated  on  the  host  but  not 


between  platforms,  (2)  positional  errors  be¬ 
tween  the  platforms  navigation  systems,  (3) 
relayed  track  reports  accumulate  errors  in 
position  and  time  since  track  reports  are 
not  time  tagged.  These  errors  result  in  the 
rule  of  not  shooting  at  a  target  not  held  on 
one's  own  sensor. 

Compensation  for  these  errors  can  be 
established  and  eventually  data  fusion  de¬ 
veloped  between  platforms,  but  it  requires 
an  extensive  communications  capability  which 
is  not  presently  available  in  a  hostile  com¬ 
bat  environment.  The  present  state  of  com¬ 
munications  as  it  relates  to  combat,  is  ad¬ 
dressing  a  simplified  threat  which  ignores 
the  multifacets  of  electronic  warfare. 

ELECTRONIC  WARFARE 

"Radio  Electronic  Combat  (REC)  is  con¬ 
sidered  a  primary  weapons  system  by  the  So¬ 
viets,  The  Soviets  have 'put  immense  effort 
and  resources  toward  exploiting  communica¬ 
tions,  radars,  data  links,  and  telemetry 
transmission  for  inteligence  in  peacetime. 
These  same  efforts  will  be  used  to  direct 
targeting  and  for  disruption  in  time  of 
war,"  states  LCDR  Guy  Thomas,  USN,  in  Mili¬ 
tary  Electronics  Countermeasures,  Dec  1982. 

There  are  many  books  and  articles  ad¬ 
dressing  the  Soviet  REC  which  results  in  an 
awareness  that  jamming  will  take  place  in 
any  conflict  -  including  jamming  of  communi¬ 
cations,  The  traditional  response  to  this 
awareness  has  been  one  of  specifying  the 
need  for  an  AJ  data  link  to  support  the  com¬ 
munications  required  for  the  single  mission 
or  a  particular  system.  Communications  so 
defined  becomes  an  auxiliary  to  the  capabil¬ 
ity  being  developed.  This  approach  to  an 
overall  weapons  system  development  cannot 
continue  if  communications  is  a  vital  need. 

A  very  basic  approach  to  antijam  tech¬ 
nology  is  to  employ  means  to  ensure  a  toler¬ 
able  level  of  interference  by  forcing  the 
tactical  j.ammer  to  spread  his  energy  over  a 
wide  portion  of  the  radio  frequency  spec¬ 
trum.  To  do  this,  the  signal  must  be  de¬ 
signed  to  take  advantage  of  procesing  gain 
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(redundancy)  or  avoidance  of  the  jammer 
(frequency  hopping).  To  be  effective,  the 
redundancy  or  movement  cannot  be  predictable 
to  a  hostile  observer  (encrypted). 

This  approach  to  AJ  development  seems 
simple  enough,  but  it  is  only  one  of  the  pa¬ 
rameters  a  communications  system  requires. 

A  communications  system  requires  an  elec¬ 
tronic  counter-countermeasures  (ECCM)  capa¬ 
bility.  The  electronic  countermeasures 
(ECM)  threat  is  not  as  simple  as  the  "jam¬ 
mer".  The  ECM  threat  is  a  phased  capability 
of  exploitation. 

1.  First,  an  attempt  is  made  to  read 
the  message  traffic,  if  possible. 

2.  Employment  of  direction  finding  on 
signals  and  correlation  of  signal  informa¬ 
tion  is  used  for  targeting, 

3.  Close  examination  of  the  signaling 
is  used  to  establish  the  electronic  order  of 
battle  for  predicting  operations. 

4.  If  the  system  cannot  be  passively 
exploited,  attempts  will  be  made  to  spoof 
the  communications  system  by  false  messages, 
capturing  synchronization,  or  using  other 
surreptitious  methods  to  deceive. 

5.  Next,  the  destruction  of  the  user 
is  pursued  by  actually  targeting  weapons  on 
the  signal  emissions. 

6.  Interference  is  further  expanded  by 
jamming  on  observed  weakness  like  exposed 
synchonization  signals,  too  few  frequencies, 
etc.,  and, 

7.  Finally,  barrage  jamming  is  used  as 
a  last  resort. 

The  communications  that  is  to  take 
place  is  command  and  control,  voice  and 
data.  In  cooperative  engagements,  there  is 
coordination  required  between  several  com¬ 
batants  to  exchange  intelligence  and  status, 
structure  the  timing  of  events,  and  assign 
roles  for  action.  This  is  not  a  one-to-one 
bulk  data  dump  as  can  be  found  in  telemetry 
links.  What  is  required  is  an  ECCM  communi¬ 
cations  system  meeting  many  constraints. 

ELECTRO-MAGNETIC  SPECTRUM 

Returning  to  the  earlier  expression  of 
what  an  AJ  capability  means,  the  use  of  the 
redundancy  or  agility  in  reducing  the  jam¬ 
ming  effects,  one  must  address  the  impact  on 
the  frequency  spectrum.  Figure  2  is  a  sim¬ 
plified  depiction  of  the  impact  of  AJ  on  the 
UHF  spectrum  specifically.  Frame  (a)  shows 
the  traditional  use  of  this  portion  of  the 
spectrum.  Channels  of  the  band  (50  to  100 
KHz)  are  assigned  to  operational  users  as 
nets  or  circuits  and  are  dedicated  con¬ 
tinuously  in  time  until  the  communications 
plan  is  changed.  Since  the  channels  are 
fixed  in  time  and  frequency  and  the  band¬ 
width  assigned  is  close  to  the  data  band¬ 
width  required,  the  jamming  signal  can  eas¬ 
ily  be  assigned  to  the  channel  and  its  total 
energy  matched  to  the  signal  energy  as  shown 
in  frame  (b).  To  defeat  this  jamming  strat¬ 
egy,  the  redundancy  is  provided  by  frequency 
or  time  expansion  and  the  agility  is  pro¬ 
vided  by  rapid  channel  switching  as  shown  in 
frame  (c). 


What  becomes  apparent  in  frame  (c)  is 
that  the  UHF  spectrum,  shown  in  this  exam¬ 
ple,  has  been  reduced  from  four  voice  cir¬ 
cuits  to  two  voice  circuits.  Of  course  the 
real  world  problem  is  much  larger  than  this 
example.  This  shows  a  reduction  of  2  to  1, 
where  the  actual  need  for  redundancy  re¬ 
quires  a  much,  much  greater  reduction. 

The  way  we  use  the  frequency  spectrum 
has  been  wasteful.  The  assignment  of  con¬ 
tinuous  time  does  not  reflect  the  actual 
time  the  signal  is  on  the  air.  Matching 
time  of  the  signal  (signaling  rate)  to  the 
assignment  time  on  channel  has  resulted  in 
recapturing  a  number  of  the  "circuits"  that 
can  be  serviced.  This  is  called  Time  Divi¬ 
sion  Multiple  Access  (TDMA)  technology, 
frame  (d),  and  is  commonly  seen  in  communi¬ 
cations  satellite  usage  where  the  informa¬ 
tion  throughout  is  limited.  The  problem  is 
that  the  225  to  400  MHz  UHF  military  band  is 
not  so  divided  and  the  TDMA  technique  is  not 
spectrum  compatible  with  the  way  the  chan¬ 
neling  is  assigned. 

This  UHF  band  is  a  small  portion  of  the 
overall  frequency  spectrum  and  intuitively 
one  would  think  there  must  be  space  availa¬ 
ble  that  is  not  structured  adversely  to  em¬ 
ploying  ECCM  techniques.  Figure  3  shows  the 
spectrum  usage  by  Navy  aircraft  only.  What 
is  not  shown,  and  what  makes  the  problem 
more  difficult,  is  the  use  of  the  spectrum 
by  other  services,  other  nations  and  civil 
facilities. 

There  are  other  spectrum  parameters  for 
ECCM  communications  that  can  be  addressed  by 
observing  Figure  3.  Consider  the  frequen¬ 
cies  below  30  MHz.  These  frequencies  are 
used  primarily  for  extended  or  long  range 
communications  between  net  members,  but  they 
can  only  support  mid-range  data  rates.  Be¬ 
cause  of  the  long  range  attributes  of  these 
frequencies,  they  greatly  abet  the  enemy's 
ECM  exploitation  by  allowing  direction  find¬ 
ing,  targeting  and  weapons  direction.  Next, 
look  at  th^  frequencies  above  2  GHz.  In 
this  region,  the  range  of  propagation  be¬ 
comes  very  short  and  high  power  transmis¬ 
sions  (high  power  generation)  are  required 
to  overcome  the  noise  background  and  high 
attenuation  encountered.  Usually,  very  di¬ 
rectional  antennas  are  employed  to  attain 
the  required  Effective  Radiated  Power  (ERP) 
in  the  signal.  These  frequencies  can  sup¬ 
port  higher  data  rates,  but  the  directional¬ 
ity  required  does  not  support  tactical  com¬ 
bat  communications,  command  and  control, 
which  is  omnidirectional  in  nature  (broad¬ 
cast  vs,  telemetry).  More  simply  stated, 
pointing  of  antennas  does  not  fit  the  opera¬ 
tional  connectivity  requirements. 

What  is  left  are  the  bands  shown  in 
Figure  4  from  108  MHz  to  1.5  GHz.  As  can  be 
seen,  fixed  frequency  assignments  are  high 
in  this  region.  The  Joint  Tactical  Informa¬ 
tion  Distribution  System  (JTIDS)  has  taken 
advantage  of  the  available  time  domain  in 
the  Lx-band  by  employing  a  pulsed  waveform 
and  TDMA.  This  has  allowed  the  introduction 
for  ECCM  communications  in  the  traditional 
TACAN/IFF  band.  The  procedures  for  making 
maximum  use  of  the  time-bandwidth  spectrum 
requires  still  further  development. 
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ECCM  AND  COMMUNICATIONS 

So  far,  this  discussion  has  adhered  to 
the  simplified  explanation  of  AJ  and  its  im¬ 
pact  on  the  frequency  spectrum.  In  order  to 
address  all  the  needs  in  defeating  the  ECM 
exploitation,  one  must  compare  the  conflict¬ 
ing  needs  for  both  ECCM  and  communications. 

There  are  certain  attributes  of  a  com¬ 
munications  system  that  the  user  takes  for 
granted.  When  the  communications  require¬ 
ments  were  not  so  great,  relying  on  teletype 
and  voice,  there  was  no  need  to  specify  cer¬ 
tain  parameters  of  the  platform  system  - 
they  were  expected. 

What  happens  to  these  common  place  re¬ 
quirements  when  ECCM  is  introduced  with  its 
attributes  of  jam  resistance,  cryptographies 
security,  and  lov/  probability  of  exploita¬ 
tion? 

Multifunction  operations  from  a  plat¬ 
form  were  handled  by  multiple  radios,  each 
switched  in  as  needed  or  dedicated  to  a  par¬ 
ticular  function  such  as  intercept  control. 

With  the  advent  of  ECCM,  multifunction 
operations  cannot  be  easily  satisfied  by 
multiple  ECCM  radios.  To  attain  AJ,  secur¬ 
ity  and  LPE,  it  is  necessary  to  treat  the 
radio  frequency  (r-f)  spectrum  as  a  system, 
the  protocols  of  access  and  sharing  are  ad¬ 
dressed  at  this  spectral  level,  not  the 
platform  or  user  level.  Under  this  ap¬ 
proach,  there  is  a  platform  terminal  that 
must  provide  simultaneous  service  to  the 
multifunctions  (track  reporting,  sensor  co¬ 
ordination,  platform  control,  voice).  Mul¬ 
tiple  terminals  on  a  platform  are  not  prac¬ 
tical.  First,  true  ECCM  is  expensive  and 
not  so  readily  afforded  as  the  simple  UHF 
radios.  Solving  the  spatial  r-f  compatibil¬ 
ity  issues  naturally  results  in  a  single 
terminal  (various  sizes)  per  platform. 

Multinetting  was  handled  by  preset 
channel  changes  (20  presets  per  airborne  UHF 
radio)  switching  from  net-to-net  as  the  ac¬ 
tion  enfolded.  Operating  in  two  nets  simul¬ 
taneously  was  solved  by  having  two  radios. 

Multiple  netting  in  ECCM  becomes  com¬ 
plex,  User  profiles  have  changed.  Air¬ 
craft,  such  as  the  F/A-18  and  E-2C,  and 
ships,  such  as  AEGIS,  have  become  multimis¬ 
sion  platforms  requiring  operations  in  au¬ 
tonomous  networks  supporting  the  various 
warfare  areas.  The  AJ  spectral  concerns  of 
ECCM  communications  are  still  prevalent  in 
meeting  these  netting  needs.  The  system 
must  be  organized  on  a  Battle  Force  or  the¬ 
ater  basis  with  each  platform  having  access 
through  a  single  terminal  to  the  spatial 
communications  system. 

Connectivity  was  one  of  broadcast. 
Anyone  on  the  circuit  or  net  could  read  the 
transmission  of  all  others. 

Connectivity  is  now  complicated  by  the 
crypto  keys  used  for  randomness  of  signals 
and  isolation  of  information  on  the  need  to 
know  basis.  By  careful  partitioning  of  cir¬ 
cuits,  the  user  need  only  observe  (decipher) 
what  is  of  interest  to  him.  Not  all  infor¬ 
mation  in  the  circuit  or  net  need  be  fil¬ 
tered  to  read  what  is  pertinent  to  each 
user.  This  partitioning  also  allows  various 
levels  of  classified  traffic  to  be  supported 
by  the  same  system  and  terminals. 


Capacity  was  usually  sufficient  and  any 
conflicts  in  use  could  be  met  with  common 
user  protocol  of  waiting  until  the  channel 
was  free. 

Capacity  must  now  be  considered  on  a 
system  basis,  much  like  satellite  communica¬ 
tions  where  the  repeating  bandwidth  is  lim¬ 
ited.  As  shown  previously,  the  AJ  aspects 
have  reduced  the  throughput  of  the  r-f  spec¬ 
trum.  To  increase  the  capacity,  we  must  now 
employ  time  division  multiple  access  tech¬ 
niques  that  again  requires  overall  systems 
control  on  a  spectral  basis.  For  low  proba¬ 
bility  of  exploitation  reasons,  this  access 
again  must  appear  random  lest  the  perceiver 
predict  the  EOB  and  then  know  when  and  how 
to  jam. 

Synchronization  was  not  even  a  concern 
with  the  modulations  employed.  The  initial 
problems  requiring  some  sort  of  synchroniza¬ 
tion  had  to  do  with  determining  which  was  a 
"one"  or  "zero"  in  pulse  code  modulation.  A 
simple  synchronization  preamble  solved  the 
problem. 

Synchronization  is  now  the  key  to  ac¬ 
cessing  the  ECCM  system.  Since  the  communi¬ 
cations  hinges  upon  a  pseudo-random  key 
stream,  there  must  be  a  way  to  acquire  this 
key  stream.  Further,  this  means  of  acquisi¬ 
tion  must  be  heavily  protected  from  the  LPE 
techniques  which  are  looking  for  predictable 
aspects  of  any  system.  This  is  expensive  to 
attain.  It  is  the  most  vulnerable  aspect  of 
ECCM  and  cannot  be  treated  lightly.  Once 
again,  this  hinges  upon  the  crypto  codes  and 
reduces  the  universality  of  access  to  the 
system. 

Ranges  required  for  tactical  operations 
now  definitely  requires  relay.  Relay  of 
course  is  replication  of  the  circuit  or 
channel  which  in  ECCM  requires  another  set 
or  access  parameters  be  added  to  the  sys¬ 
tem.  The  circuit  to  be  relayed  must  be  re¬ 
ceived  and  then  reconstituted  for  transmis¬ 
sion  on  a  separate  circuit  which  impacts  on 
system  capacity. 

With  ECCM  communications,  the  spectral 
r-f  compatibility  has  been  addressed  earl¬ 
ier,  With  the  design  of  a  good  ECCM  system, 
platform  r-f  compatibility  is  part  of  the 
single  terminal  design  and  the  biway  opera¬ 
tions  with  existing  conventional  communica¬ 
tion  equipments.  With  a  poor  ECCM  design, 
the  platform  compatibility  problem  is  virtu¬ 
ally  unsolvable  on  a  fleet  or  theater  basis. 

Joint  operations  is  still  an  opera¬ 
tional  command  and  control  problem,  even 
with  identical  ECCM  or  conventional  equip¬ 
ments,  ECCM  does  require  greater  planning 
since  spatial  operational  netting  and  cryp¬ 
tographic  codes  must  be  predetermined  and 
delivered.  Spontaneity  of  operations  is  not 
possible,  otherwise  the  system  violates  as¬ 
pects  of  LPE. 

SUMMARY 

In  summary,  this  paper  presents  only  a 
cursory  description  of  a  complex  issue  that 
must  be  addressed  in  the  design  stage  of 
systems.  When  planning  systems  that  pursue 
cooperative  tactics,  the  design  of  the  in¬ 
formation  transfer  system  is  no  trivial 
task.  As  shown  here,  the  information 
transfer  problem  is  large  and  complex,  while 
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the  r-f  spectrum  available  is  small  and 
crowded.  JTIDS,  with  its  attendant  TADIL  J, 
is  the  only  system  under  development  that 
addresses  the  command  and  control  require¬ 
ments  of  tactical  combat.  Each  platform  de¬ 
veloper  or  manager  cannot  design  his  own 
communications f  command  and  control  system 
nor  can  the  system  be  designed  re¬ 
motely.  The  communications  system  now  be¬ 
comes  an  integral  part  of  each  platform  de¬ 
sign.  The  systems  engineering  aspects  of 
the  problem  requires  the  cooperation  of  all 
users.  This  creates  a  major  difficulty  for 
communications  system  development.  Address¬ 
ing  the  communications  problem  brings  along 
the  realization  that  part  of  the  programs 
development  must  be  given  up  to  another  or 
collective  authority.  As  a  result,  the  is¬ 
sue  of  communications  is  often  put  off  and 
not  considered  until  the  program  is  com¬ 
pleted.  Of  course,  at  that  time  it  is  too 
late  and  the  system,  if  fielded,  relies  on 
vulnerable  communications. 

Realization  of  the  complexity  of  the 
issue  can  possibly  lead  to  solutions:  ig¬ 
noring  the  issue  will  likely  lead  to  fail¬ 
ure  . 


FIGURE  1 

TACTICAL  MANAGEMENT  INFORMATION  SYSTEM 


FIGURE  2 

SPREAD  SPECTRUM 
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FIGURE  4 

COMMUNICATIONS,  NAVIGATION  AND  BEACON 
FREQUENCY  UTILIZATION 
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ABSTRACT 

Electronic  warfare  (EW)  systems  aboard  the  fighter  aircraft  of  the  1990'8  will  require  numerical  and  symbolic  data  processing  to  ensure  Pjf 
ity.  This  paper  discusses  an  overview  of  the  required  algorithm  which  the  EW  Situation  Assessment  function  will  perform,  its  interfaces  with  the  midtisensor 
Fusion  function  and  the  Resource/Response  Management  (R2m)  function,  and  threat  data  support  need^Specifically,  the 

will  be  discussed:  Threat  Inference,  Intent,  Lethality,  Countermeasures  (CM)  Effectiveness,  and  Risk.  Threat  Inference  augments  the  o^ut  pro  ded  y 
Fusion  in  three  areas:  threat  Identification/Classification  (ID/Class),  range,  and  the  associatiw  of  Fusion  tracks  as  a  weapon  system.  ^  ^ 

kinematics,  mode  changes,  and  multispectral  sensor  report  information  to  discern  threat  goals.  Threat  Lethality  uses  a  pnon  and  real  time  Fusion  outputs  to 
estimate  the  probability  of  kill  with  the  associated  time-to-go.  Countermeasures  Effectiveness  factors  in  the  countenneasurc  schedule  from  the  R  M  function 
and  threat  behavior  as  a  result  of  applied  countermeasures.  Threat  kinematics,  emitter  mode  dynamics,  and  a  pnori  countermeasures  effectiveness  are  used  to 
evaluate  the  reduction  in  threat  risk.  Threat  Risk  is  shown  to  be  a  nonlinear  function  of  Intent.  Lethality,  and  Countermeasures  Effectiveness.  The  paper 
describes  how  Kalman  filtering  and  EW  Expert  rules-of-thumb  comprise  the  basis  of  performing  EW  situation  assessment. 
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ELECTRONIC  WARFARE  SITUATION  ASSESSMENT  PROCESSING  REQUIREMENTS 
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INTRODUCTION 

Electronic  warfare  (EW)  systems  aboard  the  fighter  aircraft  of  the 
199 O’s  will  require  numerical  and  symbolic  data  processing  to  ensure 
platform  survivability.  The  charter  of  EW  is  to  maximize  mission  success 
(Pms)  while  minimizing  his  probability  of  being  shot  down 

(i.e.,  maximizing  probability  of  survival,  Ps)-  This  amounts  to  both  a 
defensive  as  well  as  an  offensive  role.  By  defining  threat  and  target,  the 
roles  of  EW  situation  assessment  become  clear.  A  threat  is  the  entity 
which  is  attempting  or  has  the  capability  of  doing  harm  to  the  ownship. 
Consequently,  the  ownship  takes  on  a  defensive  posture  to  maximize  P3  . 
A  target  is  an  entity  which  the  ownship  will  attack  or  be  offensive  to¬ 
ward. 

From  these  definitions,  it  is  readily  apparent  that  the  use  of  entity 
(threat  and  target)  location,  Identification/Classification  (ID/Class),  and 
emitter  mode  dynamics  is  vital  to  perform  EW  situation  assessment.  The 
feedback  to  situation  assessment  from  resource/response  management 
must  also  be  input  to  assist  in  closing  the  data  processing  loop  when  a 
response  is  made. 

EW  SITUATION  ASSESSMENT  REQUIREMENTS 

Figure  1  depicts  the  generic  data  processing  control  loop  aboard  the 
aircraft.  The  inputs  to  situation  assessment  include  target  kinematic  (posi¬ 
tion,  velocity,  and  acceleration)  and  attribute  (ID/Class,  mode)  informa¬ 
tion. 


The  emissions  control  (EMCON)  posture  aboard  a  fighter  may  be  ac¬ 
tive,  semi-active,  or  passive  during  different  phases  of  a  mission.  As 
such,  the  output  from  fusion  may  not  be  as  complete  during  all  important 
phases  of  the  scenario. 

Consequently,  an  inference  function  to  fill  in  the  important  missing 
data  is  required.  Once  the  fusion  track  file  has  been  augmented,  the 
intent  of  the  entity  needs  to  be  determined  (whether  it  is  hostile,  non- 
hostile,  or  in  a  transitioning  process).  Then,  given  that  the  threat’s  intent 
is  non-zero,  a  measure  of  its  ability  to  harm  the  ownship  needs  to  be 
evaluated.  Then,  given  that  the  intent  and  lethality  of  the  threat  were 
assessed,  a  method  to  compute  the  threat  risk  using  these  data  is  re¬ 
quired.  Finally,  as  countermeasures  are  applied  against  the  threat,  a 
method  of  monitoring  its  effectiveness  is  required  to  factor  this  into  the 
risk  calculation  reduction  achieved.  If  the  countermeasure  is  effective, 
the  threat  risk  is  reduced  to  an  acceptable  level.  Otherwise  the  Resource/ 
Response  manager  needs  to  schedule  alternate  assets  and/or  alert  the 
mission  computer.  Figure  2  captures  these  required  EW  situation  assess¬ 
ment  functions  in  the  form  of  a  functional  block  diagram. 

EW  SITUATION  ASSESSMENT  SUBFUNCTION 
REQUIREMENTS 

To  develop  situation  assessment  requirements,  the  EW  expert  rules- 
of-thumb  need  to  be  kept  in  mind.  These  involve  threat  data  such  as 
weapons  effectiveness  zones,  multispectral  emitters,  and  dynamics  of  the 
aircraft  and  missiles  during  an  engagement. 


Figure  1.  Responses  are  Based  on  Continuously  Updated  Awareness  of 
Total  Environment  Situation 
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Figure  2.  EW  Situation  Assessment  Block  Diagram 


Threat  inference  needs  to  augment  the  fusion  track  file  in  range,  ID/ 
Class,  and  weapon  system  categories. 

Range  inference  is  performed  to  supply  range  information  to  a  threat, 
be  it  airborne  or  land-/sea-based.  Methods  to  do  this  vary  as  functions 
of  geometry  and  assumptions  made. 

IDlClass  inference  supplies  some  form  of  ID/Class  of  the  entity  when 
no  real  time  sensor  information  is  available.  Use  of  threat  data  relative  to 
the  mission  provides  clues  as  to  probable  threats  that  are  likely  to  be 
encountered. 

Weapon  system  inference  will  thread  different  fusion  track  file  entries 
which  do  not  “appear”  to  go  together  based  on  normal  fusion  rules.  This 
should  cover  multispectral  emitters  belonging  to  a  weapon  system.  A 
weapon  system  will  include,  in  the  case  of  an  aircraft,  the  various  emitters 
onboard. 

Threat  intent  examines  real  time  fusion  track  file  data  which  provides 
clues  as  to  the  intent  of  the  threat  to  harm  the  ownship.  It  will  piece 
together  the  kinematic  history  (aspect  of  the  threat  relative  to  the 
ownship  in  the  case  of  an  airborne  threat),  the  RF  emissions  (ID/Class 
and  node),  and  the  complementary  multi-spectral  weapon  system  com¬ 
ponents. 

Threat  lethality  evaluates  the  ability  of  the  threat  to  kill  the  ownship 
given  that  its  intent  is  nonzero.  It  will  need  to  use  threat  platform  ID/ 
Class  and  state  vector  information  to  compute  probable  weapon  (s)  to  be 
launched,  their  flyout  characteristics,  and  the  assessed  resultant  PkILL 
given  the  fact  that  a  weapon  was  used. 

For  land/sea  threats,  the  cumulative  PkILL  n^ust  be  determined  based 
on  ownship  speed,  altitude,  and  heading  relative  to  the  netted  threat 
laydown  forseen  to  be  encountered. 

Countermeasures  effectiveness  will  factor  in  the  threat  and  applied 
countermeasure  (s)  to  assess  whether  threat  risk  has  indeed  been  reduced 
by  using  fusion  output  and  countermeasure  ID. 

This  is  assessed  by  monitoring  RF  emitter  modes  and  kinematics  of 
missiles  in  fli^t.  For  the  missile  case,  some  form  of  Kalman  filter  design 
is  required  as  outlined  in  Figure  3. 

The  key  role  of  the  Kalman  filter  in  EW  situation  assessment  resides  in 
countermeasures  effectiveness.  The  modeling  of  missile  dynamics  affects 
state  vector  estimation  accuracy.  The  Singer  [1]  model  was  explored  and 
is  summarized  in  Figure  4. 

In  this  model,  the  target  acceleration  a(t)  is  modeled  as  a  zero-mean 
random  process  with  exponential  autocorrelation.  In  Figure  4,  is 
the  variance  of  the  autocorrelation  and  l/QJ  is  the  time  constant  of  the 
target  acceleration.  The  target  can  accelerate  at  a  maximum  rate 
A^MAX  (""A  max)  ^  probability  Pm  AX' 

accelerate  between  the  limits  -Am AX  AmaX  according  to  the 

appropriate  uniform  distribution.  For  missiles,  this  model  was  modified  to 
accommodate  the  non-linear  velocity  versus  time  profile  as  shown  in  Fig¬ 
ure  5. 


Figure  3.  Missile  Endgame  CM  Effectiveness  Incorporates  All 
Available  Sensor  Kinematic  and  Attribute  Information 

Using  the  modified  Singer  model  has  led  to  Kalman  filter  designs  which 
do  not  diverge. 

Threat  risk  is  a  combination  of  the  threat  intent,  lethality,  and  coun¬ 
termeasures  effectiveness.  The  manner  in  which  to  combine  these  pa¬ 
rameters  may  vary.  One  way  to  approach  the  calculation  follows. 

When  the  threat  intent,  lethality,  or  both  is  zero,  then  the  risk  should 
also  be  zero.  If  the  intent  and  lethality  are  non-zero,  then  a  product  of 
the  two  is  a  way  of  combining  the  two.  So,  threat  risk  without  counter¬ 
measures  applied  can  be  written  as  a  product  of  intent  and  lethality. 
When  countermeasures  are  applied,  the  EW  system  has  effectively  closed 
the  loop  against  the  threat.  Consequently,  the  risk  without  applied  coun¬ 
termeasures  needs  to  be  attenuated  accordingly.  This  leads  to  the  follow¬ 
ing  expression  for  threat  risk: 

Risk  =  Intent  x  Lethality  x  CM  Effectiveness  (1) 

SIMULATION  RESULTS  -  AIRCRAFT  AND  MISSILE  SALVO 
CASE 

Three  entity  types  are  tracked:  an  advanced  air-to-air  missile,  sur¬ 
face-to-air  missile,  and  threat  aircraft  (which  fires  the  air-to-air  missile) . 

The  engagement  is  a  low  altitude  scenario  slice  where  the  AAM  and 
SAM  form  a  salvo  against  the  ownship. 

Figures  6,  7,  8,  and  9  give  normalized  velocity  and  range  data  for  the 
air-to-air  missile  (AAM)  and  surface-to-air  (SAM)  missile  threats,  re¬ 
spectively. 
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DISTANCE  (METERS)  x  1000  VELOCITY  (m/s) 


Figure  4.  Singer  Target  Acceleration  Model 


When  a  missile  launch  is  confirmed  and  possible  missile  ID  (SA-XX  or 
AA-YY)  has  been  established,  an  accurate  Kalman  filter  model  can  be 
performed  by  using  the  a  priori  phases  (I,  II,  III)  of  missile  flight.  The 
accelerations  and  time  boundaries  then  prescribe  accurately  the  first- 
order  Markov  parameters. 

Phase  I  ■Pniax*®'®®’  ^max“  A  g’s,  alpha  =  proportional  to 

boost  time 

Phase  II  -Pfnax”  ^0“  ^max"  ®  9'®-  ==  proportional  to 

sustain  time 

Phase  III =  0.98,  PqsO.OO,  C  g's,  alpha  =  proportional  to 


Figure  S.  Three-Stage,  First-Order  MflriS:ov  Missile  Acceleration 
Model 


SCENARIO  TIME  (s) 


re  6. 


Ground  Truth  of  AAM  Velocity  and  Range  Rate  versus  Time 


Figure  8.  Ground  Truth  SAM  Velocity  and  Range  Rate  versus  Time 


Figure  7.  Range  versus  Time  for  AAM/Ownship 


Figure  9.  Range  versus  Time  for  SAMlOwnship 
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Algorithms  not  discussed  in  this  paper  were  used  to  implement  Infer¬ 
ence,  Intent,  Lethality,  Countermeasures  Effectiveness,  and  Risk,  and 
were  applied  to  this  scenario.  On-board  sensors  were  selected  and  coun¬ 
termeasures  were  applied  for  the  scenario. 

Figures  10  and  11  depict  the  risk  parameters  for  the  AAM  and  SAM. 


Figure  10.  Components  of  Risk:  AAM 


SUMMARY 

In  this  paper  we  have  done  the  following: 

•  Discussed  the  role  of  EW  situation  assessment  (SA)  in  the  filter 
of  the  1990's. 

•  Walked  through  the  top-level  SA  requirements  which  need  to  be 
performed  by  situation  assessment. 


Figure  11.  Components  of  Risk:  SAM 


•  Mapped  the  top-level  SA  requirements  into  subfunctions,  their 
requirements  and  interfaces  between  fusion  and  resource/response 
management. 

•  Discussed  how  Kalman  filtering  and  EW  expert  rules-of-thumb 
comprise  the  basis  of  performing  EW  SA. 

•  Showed  typical  risk  parameters  via  use  of  simulation  results  for 
AAM  and  SAM  threats. 
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A  statistic  has  been  devised  which  indicates,  with  reasonable  probability,  when  a  contact 
which  has  been  identified  as  some  particular  member  of  a  possible  contact  list  by  a  Bayesian 
classifier  is  in  fact  not  a  member  of  that  list  at  all,  i.^ 
hand . 


is  "alien"  to  the  database  at 


Belying  on  one’s  a-priori  knowledge  of  the  statistics  of  the  observation  process,  the 
statistic  is  the  average,  z,  of  N  mean-zero,  unit  variance  statistics,  where  N  is  the  number  of 
observations  from  the  various  sensors  employed.  If  the  contact  has  parameters  differing 
sufficiently  from  the  member  of  the  set  which  was  chosen  by  the  Bayes*  classifier,  then  the  z- 
value  will  diverge  from  zero  beyond  the  range  commensurate  with  the  number  of  observations,  and 
the  operator  is  warned  of  the  presence  of  a  new  contact  not  in  the  existing  list. 

The  statistic  is  linearly  related  to  the  conditional  entropy  of  the  stream  of 
observations.  and  combines  the  outputs  of  an  arbitrary  number  of  •  In  simulation 
studies  with  four  "sensors",  it  correctly  indicated  contacts  as  alien  of  the  time,  while 
making  only  2%  errors  in  identifying  contacts  which  were  in  the  database. 
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INTRODUCTION 

The  Bayesian  classifier  is  generally 
regarded  as  the  algorithm  most  efficient  at 
identifying  a  contact  as  a  particular  member 
of  a  list  of  possible  contacts,  given  obser¬ 
vation  data  with  some  known  statistical 
distribution.  However,  there  are  (at  least) 
three  conditions  which  can  cause  difficul¬ 
ties  with  the  process,  namely 

a)  poor  knowledge  of  the  statistics 

b)  missing  data  on  some  of  the  chara¬ 
cteristics  of  some  of  the  list 
members 

c)  a  contact  that  is  not  in  the  list 
at  all  ("alien”) 

This  paper  will  address  the  third  diff¬ 
iculty.  It  will  outline  the  computation  of  a 
statistic  which  can  indicate  the  presence  of 
an  alien  contact  by  quantitatively  comparing 
the  stream  of  observations  with  one’s  expec¬ 
tations  for  the  list  member  which  the  Bayes¬ 
ian  classifier  selects. 

This  will  enable  Bayesian  classifiers 
to  be  used  with  more  confidence,  exploiting 
their  simple  treatment  of  sensor  informa¬ 
tion,  and  will  develop  in  trained  observers 
an  additional  feel  for  the  likelihood  that 
the  contact  is  unlike  previously  encountered 
ones.  The  implementation  of  the  method  is 
straightforward,  although  it  does  rely  more 
heavily  upon  accurate  knowledge  of  the 
statistics  of  sensor  performance,  pointing 
out  again  the  necessity  for  making  measure¬ 
ments  and  historical  records  of  essential 
sensor  statistical  parameters. 

The  first  section  will  treat  the  beha¬ 
viour  of  the  product  of  the  conditional 
probabilities  of  sensor  outputs,  developing 
the  importance  of  the  logarithm  of  the  con¬ 
ditional  probability  as  a  random  variable  of 
key  importance  in  the  alien  identification 
process.  The  second  section  will  discuss 
methods  for  computing  means  and  variances  of 
these  logarithms  for  various  sensors,  in 
order  to  center  and  normalize  the  measured 
random  variable  into  the  "z-statistic" ,  a 


mean- zero,  unit  variance  random  variable 
which  can  be  combined  from  multiple  sensors 
into  an  averaged  "z"  which  can  then  be  quan¬ 
titatively  examined  to  obtain  the  warning  of 
alien  presence.  The  third  section  will  pre¬ 
sent  some  test  results,  and  discuss  the 
performance  of  the  method.  A  summary  will 
follow. 

A  STATISTICAL  ESTIMATOR  FOR  AN  ALIEN  SHIP 

The  initial  assumptions  are  that  while 
the  ships  in  the  database,  and  the  contact, 
have  unique,  well-defined  properties,  the 
^  observation  is  necessarily  corrupted  by 
transmission  losses,  observational  inaccura¬ 
cies,  and  the  like,  so  that  what  is  reported 
is  likely  not  to  be  the  exact  truth,  but 
only  something  nearby.  One  also  assumes  that 
one  has  enough  information  about  the  perfor¬ 
mance  of  the  observational  system  so  that 
one  can  calculate  the  probability  that  some 
observation  "o"  might  be  reported,  given 
that  the  contact  was  actually  ship  "s", 
P(o!s). 

In  particular,  the  paradigm  is,  say,  a 
measurement  of  the  location  of  a  .ship’s 
mast,  as  a  percentage  of  the  ship’s  length. 
Truth  for  a  given  ship  is  x  ,  and  the  obser¬ 
vation  is  reported  as  y.  One  assumes  that 
one  can  calculate  the  probability  P(y|x,dx) 
which  is  really,  given  the  discretization  of 
one’s  reporting  at  intervals  of,  say,  dx  = 
1%,  the  probability  that  the  report  will  be 
in  the  range  [y  -.005,y  +.005].  For  the 

discussions  here,  simply  assume  that  the 
probability  density  is  Gaussian,  with  a  mean 
of  X  and  a  standard  deviation  of  s,  so  that 
the  probability  "P(y}x,dx)"  is 

P(y!x,dx)  = 

dx*exp[-0 . 5*(y-x)**2/s**2]/(s  2  )  (1) 

The  actual  probabilities  are  computed  accor¬ 
ding  to  the  database  information  for  the 
particular  observation  system  and  variable 
considered,  but  do  not  change  the  considera¬ 
tions  involved  here. 
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The  Bayesian  approach  regards  the 
observations  as  necessarily  coming  from 
ship  in  the  database,  the  only  task  being  to 
assess  the  relative  likelihoods^  of  their 
having  come  from  each  ship.  This  information 
is  then  combined  with  the  prior  probabili¬ 
ties  of  the  various  ships  to  yield  the  final 
probability  that  the  contact  is  any  particu¬ 
lar  ship  in  the  dictionary,  given  the  obser¬ 
vations  : 


P( ship  lobs)  = 

P(obs;ship)*PQ(ship)/P^(obs)  (2) 

The  P  (obs)  is  determined  simply  by  nomali- 
zation,  since  one  knows  the  probabilities  in 
( 2 )  must  sum  to  1 . 0 . 

However,  in  the  case  where  there  may  be 
ships  not  in  the  database,  this  determina¬ 
tion  of  P  (obs)  is  necessarily  incorrect. 
Yet  the  Bayes’  procedure  will  select  a  ship 
(or  ships)  from  the  database  as  the  most 
likely,  and  give  one  no  warning  that  there 
is  really  something  out  there  that  has  not 
been  seen  before. 

So,  consider  the  case  where  the  Bayes’ 
process  has  narrowed  down  the  choices  to 
only  one  ship  in  the  database  as  the  likely 
contact,  because,  of  the  ones  available  to 
us,  it  was  consistently  most  like  the  obser 
vation  series.  Then  one  could  ask,  "If  ships 
may  exist  which  are  not  in  the  dictionary, 
is  the  observation  series  ’o’  probable, 
given  that  the  contact  was  the  ship^  s  ? 
I.e.,  on  an  absolute  basis,  how  big  is 
P(ols)?"  That  is,  if  ship  "s"  were  the  con¬ 
tact,  is  the  probability  P(ols)  one  in  a 
thousand?  in  a  million?  Surely  there  is  some 
value  of  P(o!s)  at  which  one  would  be 
willing  to  assert  that,  even  though  s 
looked  most  like  "o"  amongst 
dictionary,  it  didn’t  look  enough  like  it  to 
be  satisfactory  and  one  is  led  to  the  con 
elusion  that  there  must  be  something  new  out 
there . 


Unfortunately,  even  if  the  contact  .were 
■'s",  as  each  of  a  sequence  of  observations 
is  ’received,  the  associated  probability 
bounces  around.  If  one  multiplies  those 
probabilities  together  (assuming  the  obser 
vat ions  were  independent)  the  overall  proba 
bility  of  the  sequence  gets  smaller  and 
smaller,  and  one  is  led  to  trying  to  eva 
luate  whether  or  not  it  is  getting  smaller 
at  the  right  rate.  How  can  this  be  done? 

If  one  had  a  single  fraction  q  such 
that,  on  the  average,  q  were  equal  to  the 
product  of  N  observed  probabilities,  then 
for  the  i-th  observation  o . ,  one  could  take 
the  ratio  r.  of  P(o. !s)/q,^and  in  some  sense 
it  should  be  about  one.  The  correct  sense  is 
that 

Lim  [r-^ro*.  .  =  1  (3) 

N-> infinity 


If,  however,  the  observations  were 
coming  ’from  a  ship  not  in  the  database, 
which  had  a  different  value  of  x  ,  the 
probabilities  would  be  consistently  low,  and 
the  product  of  the  ratios  would  tend  rapidly 
to  zero.  So  the  strategy  is  clear:  one 
should  find  the  number  "q" ,  and  upon  recei^ 
ving  each  observation  o. ,  compute  r .  = 
P(o.!s)/q,  multiply  the  accumulated  product 
(initialized  at  1.0)  by  r . ,  and  if  the 
result  is  smaller  than  some  tolerance. 


declare  that  the  putative  ship  could  not  be 
the  actual  target  so  that  there  must  be 
something  new  out  there. 

Note  that,  if  the  distributions  of  the 
observations  are  different  for  the  ship  s 
and  the  unknown  contact,  it  is  possible  for 
the  ratio  to  be  greater  than  1.0  on  the 
average,  and  the  product  to  diverge  to  large 
numbers.  This  would  happen,  e.g.,  if  the  two 
ships,  "s"  in  the  database  and  "c",  the 
contact ,  had  the  same  value  of  x  but  c  s 
distribution  wore  much  narrower.  Then  on 
computing  P(ols),  one  would  be  getting  an 
excess  of  large  probabilities,  and  E{r. 
would  be  greater  than  one.  This  probably 
will  not  happen  in  our  case,  since  the  dis¬ 
tribution  is  presumably  characteristic  of 
the  observation  system  and  not  the  ship,  and 
so  would  be  identical  for  both  "s"  and  "c'  . 
Then,  if  the  ship  parameters  are  the  same, 
the  two  contacts  will  look  identical  to  the 
system,  and  so  no  discrimination  is  possi¬ 
ble.  In  a  multi-sensor  system,  however, 
there  will  generally  be  different  values  of 
some  variables,  so  that  the  ratio  will  still 
tend  to  zero  for  a  non-database  (alien) 
ship. 


Operationally,  one  has  difficulties 
with  the  product  formulation  of  this  sequen¬ 
tial  test,  so  instead  one  considers  the 
logarithm  of  the  product,  since  this  is 
simply  the  sum  of  the  logarithms  of  the 
individual  ratios.  The  N-th  root  of^^  the 
product  is  then  the  average,  and  "q"  is 
given  by  the  expectation  of  the  logarithm  of 
the  probability, 

InCq]  =  E{ln[P(ols)l}  (4) 

which,  apart  from  a  constant  multiplicative 
factor,  is  the  conditional  entropy  of  the 
observation  stream. 

The  logarithm  of  the  ratio  r.  can  then 
be  normalized  to  a  statistic  z^,  where 

z.  =  k*{ln[P(o. Is)]  -  ln[q]}  (5) 

This  is  a  mean-zero  variable,  and  if  k  is 
chosen  so  that  the  variance  is  one,  it  can 
be  averaged  with  other  z’s  (perhaps  from 
different  sensors)  to  create  a  sequential 
test  statistic  (the  cumulative  average) 
which  tends  to  be  Gaussianly  distributed 
with  mean  zero  and  variance  1/N.  Thus  one 
can  place  probabilistic  bounds  on  the 
averaged  z,  and  reject  the  hypothesis  that 
the  observations  could  have  come  from  ship 
•*s",  with  any  desired  degree  of  confidence. 

It  is  instructive  to  compute  q  for  the 
case  of  the  multivariate  Gaussian  distribu¬ 
tion.  Let  the  mean  and  covariance  matrix  for 
the  random  vector  x  be  m  and  B,  and  let  the 
inverse  of  B  be  L.  Then  the  statistic  z 
becomes 

z  =  1.0  -  k*(s-ffi)''*L*(^-ia)  (6) 

where  (s-m)~*L*(.S-m)  is  the  ^lahalanobj^ 
distance  associated  with  the  measurement  5. 
Thus,  the  central  quantity  z  for  Gaussianly 
distributed  observations  is  nothing  more 
than  the  mean  of  the  Mahalanobis  distances, 
and  this  is  compared  with  the  expected  value 
of  1.0.  As  the  number  of  observations  N 
increases,  the  distribution  of  the  mean  z 
tends  rapidly  to  a  Gaussian,  with  a  variance 
of  1/N. 
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(12) 


One’s  task,  then',  is  to  compute  the 
mean  ln[q. ]  and  standard  deviations  d.  of 
the  various  lnCP(o. { s) ] ’ s ,  so  that  ^upon 
receiving  a  particular  observation  o. ,  one 
can  compute  ^ 

z.  =  {ln[P(o.!s)]  ~  ln[q^]}/d.  (7) 

and  average  it  with  the  previous  ones,  and 
test  to  see  if  that  average  is  be/ond  the 
bounds  expected  for  the  number  of  observa¬ 
tions  gathered  so  far.  If  it  is,  one  can 
safely  drop  that  ship  ’'s”  from  active  consi¬ 
deration,  and  if  there  is  no  alternative, 
declare  that  the  contact  "c”  is  not  in  the 
database, 

■COMPUTATIONAL  CONSIDERATIONS 

It  is  clear  that,  for  the  identifica¬ 
tion  of  a  contact  "c”  as  not  being  a  ship 
”s"  that  is  in  the  database,  one  will  need 
to  compute  the  average  of  many  samples  of 
the  quantities 

z  =  k*{ln[P(ols)]  -  E{ln[P(o|s)]}}  (8) 

where  P(o!s)  is  the  probability  of  receiving 
the  observation  ’’o’',  given  that  the  sender 
was  ship  "s”,  E{*}  is  an  average  over  all 

the  observations  that  might  be  received,  and 
k  is  chosen  so  that  the  variance  of  z  is 
one.  Sometimes  k  and  ln[q]=E{lnCP(o ! s ) ] }  can 
be  computed  analytically,  but  more  often, 
the  distributions  will  be  awkward  enough 
that  the  computations  will  have  to  be  done 
numerically.  Fortunately,  one  needs  only  the 
first  two  moments  of  the  variable 
ln[P(o|s)],  so  the  numerical  computation 
should  not  be  too  expensive.  There  are  a 
couple  of  issues  to  consider. 

First,  let’s  examine  the  ideal  situa¬ 
tion,  where  one  knows  the  distribution  and 
can  compute  all  the  relevant  quantities. 
Assume  a  one-dimensional  quantity  being 
measured,  which  for  the  ship  one  thinks  it 
is  is  distributed  normally  about  zero  with 
variance  one.  Then,  as  has  been  previously 
derived,  the  z-parameter  is  related  to  the 
Mahalanobis  distance,  i.e., 

z  =  (1  -  x^)/sqrt(2)  (9) 

After  averaging  a  number  N  of  independent 
observations  of  z,  one  has  a  quantity  z  with 
mean  zero  and  standard  deviation  l/sqrt(N). 
So  one  is  happy  if,  after  Bayes  selects  a 
ship,  the  average  z  for  that  ship  is  less  in 
magnitude  than  about  l/sqrt(N). 

However,  assume  that  the  contact  was 
really  a  ship  for  which  the  quantity  being 
measured  had  a  value  of  y,  not  zero,  but 
(since  the  statistics  are  due  to  the  mea¬ 
surement  system)  distributed  with  the  same 
standard  deviation.  Then  one  finds  that  an 
average  of  N  z’s  computed  from  formula  (1) 
will  have 

E{a}  =  -y^/sqrt(2)  (10) 

Slid  2  4  9 

E{z^}  =  y^/2  +  (2y^+l)/N  (11) 

so  that  the  variance  of  a  is  (2y^+l)/N.  One 
wants  to  take  enough  observations  so  that 
the  standard  deviation  of  z  is  smaller  than 
its  average  value,  or  conversely,  one  can 
only  discriminate  against  ships  for  which 
the  y  value  exceeds  some  critical  size  y 
for  the  number  of  observations  that  one  has 
made.  These  imply  that  either 


N  >  (4y^+2)/y^ 
or 

y^r^  >  2(l+sqrt(l+N/2))/N  (13) 

A  little  table  of  y^^  vs,  N  is  enlightening: 


N 

y 

"^cr 

4 

1.169 

16 

0.707 

64 

0.459 

256 

0.311 

Thus,  it  is  very  difficult  to  resolve  two 
ships  whose  values  are  separated  by  less 
than  about  a  half  of  a  standard  deviation 
(William  Strutt,  Lord  Rayleigh,  was  right 
again!).  Experience  with  a  simulation,  so 
far,  is  that  some  ships  are  really  not 
separated  by  that  much,  so  that  considerable 
mis-identif ication  is  possible  without  the 
z-parameter  issuing  a  credible  warning. 

A  simple  simulation  of  the  z-parameter 
computation  ■  for  the  case  of  a  pair  of  con¬ 
tacts  is  shown  in  figure  1.  It  is  obvious 
that,  early  on,  there  is  insufficient  dis¬ 
crimination  between  the  two  sets,  but  that 
eventually  they  coalesce  into  clearly  dis¬ 
tinct  regions,  one  centered  about  0.0  and 
the  other  about  1.4  (standard  deviations). 
It  would  be  highly  improbable  that  one  could 
mistake  a  trace  from  the  second  set  as 
coming  from  the  first,  once  sufficient  obs¬ 
ervations  had  been  made. 


NUMBER  OF  OBSERVATIONS 


z-traces  figure  1 


Unfortunately,  one  doesn’t  usually  know 
accurately  the  probability  P(o}s).  Also,  the 
results  are  even  more  sensitive  to  the  de¬ 
tails  of  the  distribution  than  in  the  opera¬ 
tion  of  the  Bayes  classifier  itself.  There 
are  two  consequences  of  this: 

The  first  problem  is  that  many  of  one’s 
model  distributions  are  very  complex.  One 
must  conclude  that,  except  for  finite-dimen¬ 
sional  cases  with  known  distributions  and  a 
few  especially  simple  distributions  such  as 
unifotm  and  Gaussian,  analytic  computation 
is  useless,  and  so  one  needs  a  program  that 
computes  the  mean  and  mean-square  by  a 
Monte-Carlo  method. 

If  one  has  unclear  knowledge  of  the 
distribution,  then  the  actual  z’s  may  well 
have  a  different  distribution,  with  means 
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and  variances  both  different.  The  variance 
error  is  not  ordinarily  serious  since  it 
just  means  one  doesn't  have  accurate  limits 
on  the  bounds  for  warning,  and  one  could 
relax  those  bounds  a  bit  to  take  care  of  the 
error.  The  error  in  the  mean,  however,  is 
serious,  as  one  might  well  have  false  alien 
warnings  arising  from  one’s  computations.  So 
one  surely  must  provide  for  experimental 
data  to  replace  the  precomputed  quantities. 
That  means  that,  in  the  aircraft  (or  simula¬ 
tion  thereof),  real-time  updates  must  be 
performed  on  the  moments  of  ln[P(ols)]  so 
that  a  realistic  database  can  be  accumu¬ 
lated.  The  best  way  of  doing  this  is  not  yet 
clear,  but  it  seems  as  though  some  sort  of 
autoregressive  filter  will  allow  a  gradual 
replacement  of  theory  by  experiment.  Then 
this  will  imply  some  database  management 
problems  to  ensure  data  integrity  and 
consistency. 

The  determination  of  the  limits,  within 
which  the  averaged  z  can  lie  without  one’s 
being  concerned  about  the  possibility  that 
the  contact  is  an  alien,  must  be  made  only 
after  considerable  practical  experience  with 
actual  data.  Even  then,  the  report  must  be 
probabilistic,  and  relates  only  to  the 
probability  that  an  observation  sequence 
could  have  come  from  the  Bayes -selected 
ship,  not  the  probability  that  the  contact 
is  an  alien. 

VThat  one  comes  up  with,  in  essence,  is 
a  self -adapting  system  which  automates  a 
selection,  evaluation,  and  data-updating 
process,  and  incorporates  operating  exper¬ 
ience  in  presenting  probabilities  for 
further  operator  evaluation. 

A  program,  MONCAR,  was  created  to  ena¬ 
ble  large  numbers  of  simulated  ships  to  be 
identified  and  the  z  statistics  generated, 
while  using  the  expected  values  for  log(p) 
and  log(p)**2  generated  from  a  previous  off¬ 
line  calculation.  The  purpose  of  this  was  to 
obtain  insight  of  the  value  of  using  the  z 
statistics  in  identifying  alien  ships.  A 
switch  was  added  that  enables  the  operator 
to  specify  that  the  contact  is  "alien"  or 
"local"  (i.e.,  in  the  data  base).  If  a  ship 
is  specified  as  an  alien  then  it  will  be 
used  to  generate  the  observations,  but  will 
be  omitted  from  the  ships  available  for 
selection.  For  each  value  of  the  averaged  z 
statistic  a  corresponding  probability  of  its 
occurrence  is  generated.  It  is  expected  that 
most  of  the  ships  chosen  when  an  alien  is 
specified  will  have  small  z-probabilities , 
but  if  a  ship  is  specified  to  be  a  local 
ship,  then  the  chosen  ship  should  usually 
have  a  large  probability. 

TEST  RESULTS  AND  PLOTS 

The  expectation  was  tested  by  running  a 
combination  of  four  sensors  with  features 
consisting  of  radar  antenna  positions,  mast 
height,  mast  position,  and  two  superstruc¬ 
ture  profile  heights  as  measured  by  a  simu¬ 
lated  ISAR,  and  ESM  information.  Each  of  a 
list  of  100  ships  was  selected  as  local  10 
times  for  a  total  number  of  1000  trials.  The 
z-probability  data  was  sorted  and  plotted. 
Then  the  test  was  repeated  with  each  ship 
selected  as  an  alien.  From  the  plotted  data 
a  probability  threshold  could  be  set  for 
declaring  a  ship  as  a  probable  alien  with  a 
given  confidence.  For  example,  in  figure  2, 
one  can  see  that  if  one  chooses  to  regard 
all  ships  with  z-probabilities  greater  than 


0.05  as  local,  and  less  than  0.05  as  alien, 
about  2%  of  the  time  this  will  be  an 
erroneous  decision  for  ships  that  are  in  the 
database.  However,  about  58%  of  the  non¬ 
database  ships  will  be  correctly  identified. 
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SUMMARY 

A  statistic,  z,  the  mean  of  the  cen¬ 
tered  and  normalized  logarithms  of  the  con¬ 
ditional  probabilities  of  the  observations 
in  a  sequence,  has  been  identified  as  useful 
in  discriminating  between  contacts  in  one’s 
database  and  alien  contacts.  The  computation 
of  this  statistic  requires  knowledge  of  the 
distributions  of  each  type  of  obsepation, 
from  which  one  can  compute  the  logarithm  of 
the  probability  of  the  sample  observations, 
given  the  ship,  ,  as  well  as  the  mean  and 
variance  thereof.  Upon  identification  of  a 
contact  by  the  Bayesian  classifier,  the 
probability  of  one’s  having  accumulated  that 
particular  z  is  checked  to  see  if  it  is 
within  some  preset  tolerance,  '  consistent 
with  one’s  desired  level  of  risk,  and  the 
contact  is  then  reported  as  local  or  alien. 
In  one  series  of  simulations,  some  58%  of 
the  alien  ships  were  correctly  identified  as 
such,  with  only  2%  of  the  local  ships  being 
misidentified.  The  z-statistic  thus  appears 
to  be  a  useful  adjunct  to  a  Bayesian  class¬ 
ifier,  particularly  when  one  wishes  to  limit 
the  data  base,  yet  be  warned  when  a  non¬ 
database  contact  is  encountered. 
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An  Automated  Tactical  Situation  Assessment  Methodology: 

Its  Limitations  and  Impact  Upon  Tactical  Command  and  Control 
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Since  1984,  basic  research  efforts  have  been  ongoing  to  provide  Ae  U.  S.  Army  Division/CORPS 
Staffs  with  automated  assistance  during  the  preparation  of  a  tactical  situation  assessment  (TSA).  The 
current  TSA  research  paradigm  assumes  that  the  generation  of  a  TSA  is  a  problem  solving  process  wherein 
ongoing  tactical  plans  are  formulated  and  recognized.  The  TSA  tactical  plan  recognition  paradigm  makes 
use  of  Wilensky’s  relational  language,  KODIAK,  to  express  high-level,  generic,  enemy  plans.  Low-level 
KODIAK  Actions  are  expressed  using  Schank-type  scripts  which  are  believed  to  be  stereotypic  and 
extensible  thus  satisfying  the  research  requirement  for  a  domain  independent  ^nd  extensible  TSA 
paradigm.’  This  paper  argues  that  the  efficacy  of  the  plan  recognition  methodology  is,  however,  uncertain 
since  each  KODIAK  Action  script  represents  a  highly  abstracted  tactical  plan.  The  mathematical  basis  for 
a  plan  structure  and  a  plan  abstraction  transformation  is  formally  developed.  It  is  shown  that  each 
KODIAK  Action  must  be  represented  as  either  a  deterministic  plan  or  as  the  union  of  a  finite  number  of 
deterministic  plans.  The  paper  concludes  by  examining  the  behavior  of  the  TSA  paradigni,  assuming  that 
the  KODIAK  Actions  are  improperly  represented.  It  is  shown,  assuming  a  uniform  probability  of  detecting 
pre  and  post  Action  states,  that  the  TSA  paradigm  will  tend  to  converge  to  either  an  unexceptional  or  worst 
case  scenario.  The  impact  of  the  described  TSA  methodology  on  the  Army  Command  and  Control  process  is 
examined,  and  suggestions  for  improving  the  plan  recognition  paradigm  are  offered.  Finally,  experimental 
evidence  exists  which  appears  to  support  the  author  s  theoretical  results. 
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1.  INTRODUCTION 


Since  1984,  basic  research  efforts  have  been  ongoing 
to  provide  the  U.  S.  Army  Division/CORPS  Staffs  with 
automated  assistance  during  the  preparation  of  a  tactical 
situation  assessment  (TSA).  Both  contractual  and 
in-house  research  efforts  have  been  involved  in  this  effort. 
A  "how  to  perform"  TSA  paradigm  has  been  developed 
under  U.S,  Government  contract  by  The  Analytic  Sciences 
Corporation  (TASC).  Work  is  presently  underway  to 
implement  and  test  these  ideas  [Hatfield].  As  with  most 
new  ideas,  this  paradigm  has  been  received  with  some 
skepticism  and  criticism.  Hopefully,  this  paper  represents 
a  part  of  that  healthy  process  of  peer  review  and,  as  such,  is 
a  critique  of  the  project's  plan  recognition  paradigm.  Of 
particular  concern  to  the  author  are  the  possible  TSA 
analysis  errors  which  may  develop  when  improperly 
abstracted  plans  are  used  as  a  part  of  the  automated  TSA 
plan  recognition  process. 

This  paper  begins  with  a  brief  discussion  of  how  a 
rapidly  changing  technology  has  affected  the  TSA  process 
during  the  20th  Century.  This  discussion  concludes  with  a 
commentary  on  the  problems  currently  facing  a  military 
Staff  during  the  preparation  of  a  TSA.  Next,  the 
mathematical  basis  for  plan  abstraction  is  developed. 
The  TASC  TSA  plan  recognition  paradigm  is  described 
and  critiqued.  The  paper  concludes  with  a  discussion  of  a 
possible  situation  analysis  paradox  which  may  result  if 
improperly  abstracted  plans  are  implemented  within  the 
proposed  plan  recognition  paradigm.  The  effects  of  this 
analysis  paradox  on  the  Army's  Tactical  Command  and 
Control  process  are  also  examined.  Finally,  a  number  of 
possible  improvements  to  this  paradigm  are  described. 


A.  The  Effect  of  Technology  upon  the  Tactical 
Situation  Assessment  Process 

Technology  has  been  the  driving  force  changing  the 
methodology  of  warfare.  Although  the  high-level 
objectives  of  war  (e.g.,  to  occupy  and  control)  have 
remained  essentially  unchanged  since  Biblical  times,  the 
methods  used  to  implement  these  tactical  objectives  have 
changed  considerably.  Technological  innovation  and 
growth  continues  at  an  exponential  rate.  The  effect  of 
technology  upon  the  art  of  warfare  has  forever  changed 
the  TSA  process  in  three  fundamental  ways. 

First,  and  most  importantly,  technology  has  caused 
the  TSA  processing  time  to  shrink.  The  maximum  time 
permitted  to  react  to  a  tactical  threat  has  generally 
decreased  with  each  military  technological  improvement. 
For  example,  during  World  World  I  global  armies  faced 
one  another,  deeply  entrenched,  along  a  slowly  moving, 
imaginary  line  called  a  Front.  In  this  seige-like 
condition,  there  was  ample  time  to  amass  tactical 
intelligence  and  the  resulting  TSA  process  was  developed 
slowly.  Commanders  and  their  troops  came  to  know  and 
judge  their  opponents  quite  well.  With  experience,  the 
weather  and  terrain  conditions  affecting  tactical  mobility 
and  performance  could  be  accurately  predicted.  These 
conditions  essentially  remained  constant  until  technology 
developed  a  machine  capable  of  rapidly  and  safely 
traversing  enemy  front  lines:  the  tank.  Tanks  crushed  the 
immutability  of  the  Front  and  forever  changed  the 
methodology  of  warfare.  Rapidly  moving  tanks  and  forces 
demanded  that  the  TSA  process  be  performed  more  quickly 
than  previously.  Today,  the  ICBM  has  placed  virtually 
every  city  in  the  world  within  twenty  minutes  of  total 
destruction.  We  live  in  a  global  village  where  our  units  of 
time  continue  to  shrink  with  each  technological  advance. 
TSA  processing  must  now  be  performed  in  minutes  rather 
than  days. 
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Secondly,  technology  ha&  increased  the  penalty  for 
TSA  error.  We  live  in  a  time  of  Strategic  Deterrence  where 
the  threat  of  global  confrontation  is  too  horrible  to 
contemplate.  However,  for  those  with  nothing  to  lose,  the 
threat  of  Nuclear  War  is  more  rhetoric  than  fact.  With 
stakes  this  high,  accurate  and  timely  vigilance  has  become 
a  neccessity.  In  tactical  situations,  TSA  error  can  foster  a 
deadly  consequence. 

Thirdly,  technology  has  increased  our  dependence 
upon  automation.  For  both  of  the  reasons  cited  abovei  the 
rapid  shrinkage  of  situational  analysis  reaction  time  and 
the  (potentially)  enormous  penalty  paid  for  analysis  error, 
the  Intelligence  Officer  and  his  Staff  require  assistance. 
In  addition,  in  Army  21  [Army  21],  the  U.S.  Army  predicts 
that  future  battles  will  be  fought  in  more  of  a  guerrilla  or 
skirmish-like  fashion.  Battles  will  likely  be  loosely 
coordinated  with  more  responsibility  given  to  lower 
echelon  commanders.  The  Army  Intelligence  Officer  must 
be  prepared  to  rapidly  perform  a  TSA  in  a  foreign, 
unfamiliar  part  of  the  world,  often  against  an  unknown  or 
unfamiliar  enemy  commander.  Curiously,  technology 
appears  to  offer  the  only  substantive  assistance  to  the 
Army  Intelligence  Officer. 


II.  AUTOMATED  TACTICAL  SITUATION 
ASSESSMENT 

Technology  offers  assistance  to  an  Intelligence 
Officer  in  the  form  of  computer  automation.  The  promise 
of  automation  is  found  in  the  computer's  ability  to  rapidly 
process  data.  With  speed  comes  an  apparent  expansion  of 
time.,  which  is  exactly  what  an  Intelligence  Officer  needs: 
more  time  to  analyze  the  tactical  data  and  more  time  to 
react  to  an  enemy  threat. 


A.  Plans 

Basically,  the  computer  can  assist  an  Intelligence 
Officer  if  it  is  able  to  perform  the  following  tasks: 
accurately  recognize  an  enemy  tactical  planj  estimate  the 
probable  evolution  of  the  tactical  plan;  and  determine  the 
probable  threat  oi  the  evolved  tactical  plan  to  friendly 
forces.  The  key  element  of  this  automated  process  is  the 
recognition  of  a  tactical  plan.  A  tactical  plan  [Wilensky, 
1983;  Hatfield;  Chubb]  is  defined  as  a  purposeful,  rational 
sequence  of  actions  which,  when  executed  by  some  tactical 
force(s),  results  in  a  tactical  objective  or  goal,  G,  being 
realized.  A  plan,  P,  is  represented  as  a  time-contiguous 
sequence  of  tuples  each  of  which  is  executed  by  a  plan 
actor,  AC.  That  is, 

P  =  {{A,CS}i,....{A,CS}n}  (2.1) 

where  Aj  is  some  action  performed  by  AC, 
and  CSj  is  the  context  state  which  AC 

believes,  is  necessary  to  initiate  execution 
ofAj, 

for  i  =  I,...,n. 

For  purposes  of  this  discussion  and  unless  otherwise 
specified,  the  plan  actor,  AC,  is  assumed  to  be  human. 
Every  plan  tuple  {A,CS}i  ^  P  contains  two  related 
elements:  an  actiony  Aj^  and  an  associated  context  state^ 


CSi.  Each  CSjis  a  description  of  context  features  or 
conditions  which  the  actor,  AC,  believes  musthe  present 
within  the  domain  to  successfully  initiate  the  execution  of 
Ai  [Chubb].  A  plan  P  is  executed  by  an  actor,  AC,  by  the 
actor  sequentially  executing  plan  tuple  actions.  AC  begins 
by  executing  A j  within  context  state  CSj.  At  the 
completion  of  this  action,  a  new  (possibly  different) 
context  state,  CS2,  results  from  AC  executing  Ai  in  CS/. 
Action  A2  is  now  executed  within  CS^.  This  process 
continues  (if  the  plan  is  successful)  until  the  last  plan 
tuple  results  in  a  context  state  expression  which  contains 
some  plan  goal  state,  G.  That  is, 

Gp  =  AC(P)  (2.2) 

where  G/>  is  read  to  mean  "the  goal  G  of  some  plan  P". 

Familiar  examples  of  plans  are  found  in  everyday  life.  The 
sequence  of  actions  which  we  use  in  the  morning  as  we 
prepare  for  the  day's  work  represent  the  tuple  elements  of 
a  plan  called  "Get  ready  for  work"  .  Although  most  plans 
are  performed  unconsciously  and  by  rote,  plan  tuple 
execution  is  based  upon  two  Axioms. 

If  Pis  a  plan  where  P  =  {{A,CS}/,....{A,CS}n.},  then  for 
every  i,i  =  2,...,n 

Plan  Execution  Axiom  1:  Every  tuple  {A,CS}i  £  Pis 
causally  SLTid  temporally  reloted  to  its  (i-1)  and  (i  +  1) 
tuple  neighbors,  and, 

Plan  Execution  Axiom  2:  Plan  tuple  subelements,  Aj  and 
CSi,  are  variables  which  are  dynamically  developed  by 
AC.  (2.3) 

Contiguous  plan  tuples  are  causally  related  since  the 
completion  of  the  execution  of  the  ith  action,  Aj,  is 
intended  to  result  in  context  state  CSi-f-i.  If  the  context 
state  which  results  from  the  execution  of  Aj  in  CSj  by  AC 
does  not  equal  the  predicted  CBl+u  then  the  respective 
values  of  Ai,  A,>2,  and/or  predicted  context  state  CSj+i 
may  be  dynamically  changed  by  AC.  This  (human)  ability 
to  dynamically  change  or  alter  a  plan  in  accordance  with 
contextual  changes/queues  accounts  for  the  richness  of  the 
plan  expression  and  its  endless  variety.  Some  plan 
interactions,  however,  are  so  well  understood  that  the  plan 
actor  is  able  to  accurately  predict  the  interaction  of  each 
tuple  within  the  entire  plan  . 

Definition  2-1:  A  plan  Pis  defined  to  be  deterministic  if 
and  only  if  for  every  plan  tuple  {A,CS}i  e  P,  i  =  n, 
rankE{A,CS}i  —  1. 

The  notation  used  to  describe  plan  tuple  execution 
with  its  resulting  tuple-to-tuple  interaction  is: 

E{A,CS}i  ^CSi  +  i  (2.4) 

where  the  function  E  is  read  to  mean  "Ai  executed  within 
context  state  CSi  yields  the  singleton  tuple  (anticipating 
the  Aii-i  action)  context  state  expression  CSi+j".  As  a 
singleton  tuple,  CSi-f  ;  =  {CSi  +  i}  and  we  sometimes  write 
E{A,CS}i  =  {CSi  .m}  which  is  equivalent  to  (2.4). 
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For  any  successful  plan  P,  E{A,CS}n  ->  CSn-hl  where 
Gp  e  CSn-fi  and  {A,CS}/iis  assumed  to  be  the  last  tuple 
element  in  P. 

The  plan  tuple-to-tuple  relationship  in  (2.4)  may  also 
be  represented  using  a  directed  graph  tree  representation 
[Nilsson].  Each  tuple  is  represented  by  a  node  while  the 
E{Aj,CSj}  -*  CS  i  +  2  relationship  is  represented  by 
branches  or  arcs  between  nodes.  The  function  E  develops 
context  state  possibilities  resulting  from  the  execution  of 
Ai  by  AC  in  context  state  CSj,  The  branches  represent  the 
causal  and  temporal  flow  from  plan  tuple  to  tuple.  The 
tree  representation  of  E{Ai,CSi}  for  every  tuple  in  P  is 
called  an  expanded  form  tree  representation,  Pep,  of  plan 
P.  Pff  represents  every  possible  action  and  context  state 
relationship  in  one  representation  for  one  plan  (Figure 
One). 

We  define  M  to  be  the  number  of  nodes  in  P£f.  Fora 
deterministic  plan,  M deterministic  ~  n.,  where  n  is  the 
number  of  tuples  in  plan  P,  However,  not  every  plan  is 
deterministic.  Some  plans  contain  tuples  such  that 
E{A,CS}i  is  not  single-valued.  For  example,  consider  the 
Ai  —  "flip  a  coin".  The  context  state  expression  resulting 
from  E{A,CS}i  may  contain  two  possible  states:  heads  or 
tails.  The  number  of  possible  contextual  state  results  for 
E{A,CS}i  is,  however,  finite. 

Definition  2-2:  A  plan  P  is  defined  to  be  N-deterministic  if 
and  only  if  for  every  plan  tuple  in  P,  i~l,...,n, 
rank  E{A,CS}i  ~  h!  where  N  >  1  and  finite  and  there 
exists  at  least  one  plan  tuple  {A,CS}i  £  Psuch  that 
rank  E{A,CS}i  =  W. 

Notation  used  to  indicate  that  the  rank  of  the  expected 
value  of  an  ith  plan  tuple  is  greater  than  one  is: 

Ey{A,CS}i  {OSu  ,...,CSu-hj-i}  for  all  j  >  1. 

The  execution  function  Ej  here  produces  a  set  of  context 
state  tuples  which  could  also  be  written  as, 

Ey  {A,CS}e  {{CS^}  ,...,{CSu  }}. 

A  least  upper  bound  for  the  value  of  .deterministic  is, 
n 

1  +  S[maxfranfrE{A,CS}i7x  max  [rank  E{A,CS}i.i]] 

^=-2  (2.5) 

where  n  is  the  number  of  tuples  in  P. 

Plans  which  are  neither  deterministic  nor 
N-deterministic  are  defined  to  be  nondeterministic. 

Definition  2-3:  A  plan  P  is  defined  to  be  nondeterministic 
if  and  only  if  there  exists  some  plan  tuple,  {A,CS}i  e  P 
such  that  rank  E{A,CS}i  is  not  finite.  Notation  used  to 
indicate  a  countable  number  of  possibilities  is  E  or, 

E{A,CS}i  {CSz,..l 

Nondeterministic  plans  usually  include  actions  and 
context  states  which  are  best  described  as  random  or 
probabilistic  with  countably  many  possibilities.  The 
magnitude  of  M nondeterministic  is,  by  definition, 


infinitely  large.  Fortunately,  most  tactical  plans  of 
interest  are  N-deterministic.  The  magnitude  of 
-deterministic,  however,  can  be  far  too  large  to  permit 
practical  computer  implementation  of  the  plan. 


III.  COMPUTER  MODELS  AND  LEVELS  OF 
ABSTRACTION 


For  automation  to  be  an  effective  plan  recognition 
device,  one  must  be  able  to  describe  a  tactical  plan  in 
sufficient  detail.  It  is  quite  easy  to  describe  most 
deterministic  plans.  Powerful  computer  programs  exist 
which  are  able  to  accurately  recognize  (and  create)  some 
N-deterministic  plans.  For  example,  Samuel’s  checker 
playing  program  is  able  to  play  checkers  at  least  as  well  as 
any  human.  The  reason  for  this  is  that  the  size  of  the 
expanded  form  representation,  Mcheckers,  for  this 
computer  plan  is  small  enough  to  be  fully  expressible 
within  computer  memory.  As  a  result,  Samuel's  program 
can  "look  ahead”  in  the  expanded  form  tree  representation 
and  examine  alternative  context-states,  g^ven  some 
checkers  move  action  (i.e.,  E/{Ai,CSi}).  The  game  becomes 
fully  deterministic  and  the  best  move  is  always  made.  The 
program  never  loses;  at  worst,  the  result  is  a  draw. 

The  game  of  chess,  on  the  other  hand,  is  an  example 
of  an  N-deterministic  plan  where  the  magnitude  of  Mchess 
is  far  too  large  to  be  fully  represented  within  computer 
memory.  Recent  cognitive  research  [Katter]  suggests  that 
although  humans  lack  the  ability  to  accurately  recall  the 
details  of  very  large  Af  valued  plans,  they  are  able  to 
rapidly  recognize  these  plans.  These  findings  have 
suggested  the  following: 

Plan  Abstraction  Axiom:  Plans  with  large 
^expanded  form  values  can  be  faithfully  represented, 
through  a  process  of  plan  abstraction,  in  a  plan  form 
such  that  Mabstracted  <  <  Mexpanded  form- 

This  axiom  assumes  the  existence  of  some  type  of 
transformation  which  maps  (abstracts  or  reduces)  the 
expanded  form  representation  of  P  into  an  abstracted  and 
implementable  plan  form. 

A.  Plan  Abstraction 

We  begin  with  the  Pee  representation  of  an 
N-deterministic  plan  P  (Figure  One).  By  definition,  every 
tree  node  has  at  most  a  single  tuple  successor  called  its 
F ather  node.  That  is,  if  {A^CS}j  e  Pee,  then  there  exists  a 
Father  function  such  that, 

either  faf/>er{A,CS}y  =  {A,CS}/*  c  Pee, 

or  Father{ACS}j  =  0.  (3.1) 

where  {A,CS}f  ^  {A,CS}j,  {A,CS}/‘  e  Pff, 
and  {A,CS}j  t  Ey{A,CS}/*. 

The  Pff  tree  root  node  is  represented  by  the  first  plan 
tuple  in  P,  {A,CS}j.  By  definition,  tuple  {A,CS}i  is  the 
only  node  in  Pff  such  that  fat/ier{A,CS}i  =  0, 
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Without  loss  of  generality,  we  also  assume  that  the 
plan  P  has  concluded  successfully  and  that  there  exists 
some  non-empty  subset  of  terminal  nodes  C  which 
contain  the  desired  state,  Gp.  We  will  only  examine 
successful  plan  strategies  by  recursively  removing  all  of 
the  terminal  nodes  in  Pfp  which  do  not  contain  Gp. 
Without  loss  of  generality,  Pfp  may  be  represented  as  the 
set  of  remaining  successful  plans.  That  is,  each  tree 
terminal  node  in  Pff  represents  a  plan  tuple  for  a 
successfully  concluded  variation  of  plan  P. 

Root  Node:  {A,CS}i 


{A,CS}2^  {A,CS}2^  A,CS}2^ 


Terminal  Nodes:  {cSkl 

Figure  One:  Expanded  Form  Pf  p  of  plan  P 

Definition  3-1:  Define  the  set  Wj  C  Pbf  to  be: 
for  every  terminal  node  {A, CS}j  e  Ppp,  let 

Wj  ={{A,CS}j  U  {{A,CS}g  :  {A,CS}g  e  Pbf  and 
{A,CS}g  =  Father  {A, CS}x  where  {A,CS}x£Wj}}. 

Let  Y  be  the  set  of  all  Wj  as  defined.  We  assume  that  Y  is 
non-empty  with  rank  Y  =  p  >  0.  The  following  Lemma 
may  be  easily  proved  using  Definition  3-1. 

LEMMA  3.1:  Given  the  Wj  as  defined  in  3-1.  Then, 

1.  EachWj  is  a  plan. 

2.  The  first  tuple  in  each  Wj  is  the  root  node  ofPBF^ 
Conversely,  the  last  tuple  in  each  Wj  is  thejth 
terminal  node  in  Pfp. 

3.  U  Wj  =  Pbf 

4.  r\Wj  ^  0 

5.  Every  Wj  is  well-ordered. 

Our  examination  of  plan  abstraction  will  proceed  as 
follows.  First,  we  will  restrict  our  study  domain  to  the 
elements  of  Y.  We  will  examine  the  set  of  plan  abstraction 
transformations  which  map  tuples  from  Wj  and  Wfeto  a 
single  plan  tuple.  Next,  we  will  examine  the  set  of  plan 
abstraction  transformations  which  map  tuples  from  a 
single  Wj  to  itself.  Finally,  we  will  show  that  these  plan 
abstraction  transformations  may  be  used  to  reduce 
(abstract)  appropriate  subsets  of  Ppp. 

We  begin  by  describing  two  relations,  Rg  and  R  ron 
the  set  S  of  all  plans  executed  by  actor  AC. 


Definition  3-2:  Let  Bbea  partition  ofS.  Define  the  relation 
Rg  —  {(^*y)  '  ^  and  y  are  plans  in  S;  there  exists  a  class 
Ce  B  such  that  X, ye  Cif  and  only  if  x  and  y  share  the 
same  actor  goal,  G}. 

LEMMA  3.2:  Rq  is  an  equivalence  relation  on  S. 

Proof:  Obviously  Rg  is  reflexive  (xRgx)  and  symmetric 
(ifxRGy,thenyRGx).  Assume  xRgY  and  yRcz  then,  by 
definition,  there  exists  classes  C,  C  e  B  with  x,y  e  C  and 
y,z  e  C’.  Since  y  e  CHC  the  classes  are  not  disjoint, 
therefore  C  =  C.  Therefore,  x,z  e  C  xRgz.  Therefore 
Rg  is  transitive,  q.e.d. 

For  every  partioned  class  C  resulting  from  Rg  there  exists 
Pbf  e  C  and  set  V={Wj:Wj  a  successful  plan  in  class  C}. 

Definition  3-3:  Assuming  that  Visa  non-empty  set,  and  B 
be  a  partition  of  Y.  Define  the  relation 
Ry—  {  (x,y)  :  X  and  y  e  Y;  there  exists  a  class  C  e  B  such 
that  x,y  e  C  if  and  only  ifx  and  y  have  equivalent  plan 
execution  times}. 

It  can  be  easily  shown  that, 

LEMMA  3.3:  Rjis  an  equivalence  relation  on  Y. 

Unless  otherwise  noted,  we  will  assume  that  any 
discussions  involving  two  or  more  Wj  tacitly  assume  that 
the  plans  are  members  of  the  same  class  defined  by  the 
relation  {(x,y) :  xRcy  3nd  xRjy  hold}.  That  is,  for  all  x,y 
under  consideration,  x  and  y 

a.  belong  to  the  actor,  AC 

b.  share  the  same  actor  goal 

c.  are  successful  plans,  and 

d.  have  equivalent  plan  execution  times. 


IV.  PLAN  ABSTRACTION  TRANSFORMATIONS 

Definition  4-1:  Let  Pbe  a  plan  with  actor  AC,  initial 
context  state  CSi,  and  plan  goal  Gp.  Assume  Fis  a 
non-empty  subset  of  Pand  Fa  plan.  Then  Ta  is  a  plan 
abstraction  transformation  ofFif  Ta  F  ->  F' where  F"  a 
plan,  rank  f  *  <  rank  F,  and  if  F'  replaces  F  in  P, 

AC(P)  ^  Gp. 

A.  Abstracting  Tuples  from  Two  Deterministic 
Plans 

We  begin  by  enumerating  the  types  of  plans  which 
can  be  abstracted  (reduced)  to  plans  of  lesser  rank.  We 
enumerate  the  possible  tuple- to-tuple  relationships  which 
may  be  formed  from  tuples  which  belong  to  two  different 
plans,  Wj  and  Wg,  where  tuple  (A,CS}j  e  Wj,  {A,CS}g  e 
Wg,  and  Wj  Wg.  Plan  tuple  variables  to  be  considered 
are: 

a)  Actions:  Aj  and  Ag 

b)  Initial  Context  States:  CSj  and  CSg, 

c)  Final  Context  States:  CSj  +  i  e  E{A,CS}j, 

CSg+i  e  E{A,CS}g. 
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For  each  of  the  above  categories  (Actions,  Initial  CS,  Final 
CS)  ,  the  tuple  variables  are  either  equal  or  not  equal. 
Hence,  there  are  23  possible  tuple  state  descriptions.  We 
will  prove  that  some  of  these  tuple  state  descriptions  can 
be  abstracted  (a  Ta  exists)  and  that  the  remaining  state 
descriptions  can  not  be  abstracted. 

For  each  of  the  possible  tuple  state  descriptions  we 
assume  that  Wj  ,  Wgare  deterministic  plans.  This 
assumption  is,  however,  not  necessarily  true.  By 
construction,  each  of  the  Wj  appears  to  be  deterministic. 
However,  given  any  two  contiguous  tuples 
{A,CS}j,  {A,CS}i-/-2  e  Wj,  either: 

1.  E{A,CS)i  ^  CSi>;  e  {A,CS}i^i 

(deterministic  relationship) 

2.  Ey{A,CS}i  {CSj-f  jf,....CS^. 

(N-deterministic  relationship) 

THEOREM  HI:  Given  the  following  tuple  state 
description  (Aj  =  Ag;  CSj  =  CSg;  CSj^i  =  CSg+i), 
then  there  exists  tuple  {A,CS}x  and  abstraction 
transformation  Thi  such  that, 

Thi  :  {{A,CS}j  ,{A,CS}g}  ^  {{A,CShh 

Proof:  By  definition,  {A,CS}j  =  {A,CS}^.  Clearly 
{{A,CS}j  ,{A,CS}^}  is  an  N-deterministic  plan  with  input 
context  states  equal  to  E2{A,CS}j^  =  {{CSj},{CS^}}  for  some 
N-deterministic  tuple  {A,CS}^  and  with  deterministic  plan 
tuple  possibilities  {A,CSy  and  {A,CS}^. 

Let  {A,CS}x  =  {A,CS}j  where  {{A,CS}^' }  a  plan  by 
definition.  Then, 

rank{{A,CS}j}  +  rank  {{A,CS}g}  >  rank  {{A, CS}x}  and 
E{A,CS}x  =  E{A,CSJ;-  =E{A,CS}^  since CS^>i  =  CSg^t, 
Therefore  Thi  is  the  required  abstraction  transformation. 

THEOREM  H2:  Given  the  following  tuple  state 
description  (Aj  =  Ag;CSj  ^CSg;CSj+i  =CSg  +  i), 
then  there  exists  tuple  {A,CS}x  and  abstraction 
transformation  TH2  such  that, 

TH2  '  {{A,CS}j  ,{A,CS}g}  ->  {{A,CS}x}. 

Proof:  {{A,CS}^  ,{A,CSy  is  an  N-deterministic  plan  with 
input  context  states  equal  to  E2{A,CS}jt  =  {{CSy},{CS^}}  for 
some  N-deterministic  tuple  {A,CS}^  and  with  deterministic 
plan  tuple  possibilities  {A,CS}j  and{A,CS}^.  Since 
E(A,CS}j  =  E{Aj  ,CS^}  =  E(  A^  ,CSj  the  actor  action  is 
independent  of  value  of  E2{A,CS}yb.  Therefore,  let 
{A,CS}x  =  (A,CS}j  where  {{A,CS}j  }  a  plan  by  definition. 
Rank  {{A,CS}j }  +  rank  {{A,CS}^}  >  rank  {{A,CS}x}  and 
E{A,CS}:,  =  E{A,CS}j  =  E{A,CS}^  since =  CS^+jr, 
as  required.  Therefore  Th2  is  the  required  abstraction 
transformation. 

THEOREM  H3:  Given  the  following  state  description 
(Aj  Ag;  CSj  =  CSg;  CSj+i  ~  CSg  +  i)^  then  there 
exists  tuple  {A,CS}x  and  abstraction  transformation  Th3 
such  that,  Th3  :  {{A,CS}j  ,{A,CS}g}  ^  {{A,CS}xh 

Proof:  {{A,CS}j  ,{A,CS}^}  is  an  N-deterministic  plan  with 
input  context  states  equal  to  E2{A,CS}/fe  =  {{CS^},{CS^}}  for 


some  N-deterministic  tuple  {A,CS}k  and  with  deterministic 
plan  tuple  possibilities  {A, CS}y  and{A,CS}^.  Since 
E(Ay  ,CS^}  =  E{A^  ,CSj}  and  Aj  Ag,  action  differences 
do  not  effect  the  executed  tuple  values, 
i.e.,CSj>i  =  CS^-^2.  Therefore,  let  {A,CS}a:  =  (A,CS}j 
with  {{A,CS}j  }  a  plan  by  definition.  Clearly, 
rank{{A,CS}j}  +  rank  {{A,CS}g}  >  rani^  {{A, CSW  and 
E{A,CS}:c  =  E{A,CS}j  =  E{A,CS}^  as  required.  Therefore 
Th3  is  the  required  abstraction  transformation. 


THEOREM  H4:  For  each  of  the  following  deterministic 
state  descriptions  there  exists  no  abstraction 
transformation  Ta  and  plan  tuple  {A,CS}x  such  that 
Ta  :  {{A,CS}j  ,{A,CS}g}  {{A,CS}x}. 

a.  (Aj  ^  Ag;  CSj  CSg;  CSj +  2  ^  CSg^i) 

b.  (Aj  ^  Ag;  CSj  =  CSg ;CSj  +  i  CSg^-i) 

c.  (Aj  =  Ag;  CSj  ^  CSg ;  CSj +  2  ^  CSg +2) 

d.  (Aj  =  Ag;  CSj  =  CSg;  CSj 2-2  ^  CSg 2- 2) 

e.  (Aj  Ag ;  CSj  CSg;  CSj 2-1  ~  CSg 2-2)- 

Proof  For  State  Descriptions  a,  b,  c,  d:  In  each  of  these 
descriptions  E(A,CS}j  E{A  ,CS}^  since 
CSj  2-1  ^  CS^  2- 1  •  Since  {{A,CS};c}  a  deterministic  plan, 
E{A,CS}x  ^  {CSj  .f  jr ,  CSg 2-1}  and  therefore  the  required 
Ta  does  not  exist. 

Proof  For  State  Description  e;  In  this  description 
E(A,CS}j  =  E{A  ,CS}^  since  CSjf-f  i  —  CSg 2- 2-  Let 
{{A,CS}j  ,{A,CS}^}  be  an  N-deterministic  plan  with  input 
context  states  equal  to  E2{A,CS}jfe  =  {{CSj},{CS^}}  for  some 
N-deterministic  tuple  {A,CS})fe  and  with  deterministic  plan 
tuple  possibilities  (A, CS}/  and{A,CS}^.  However,  since 
A/  ^  CSj  CS^,  there  exists  no  CS^  such  that 

(Aj  ,CS2:}  =  {A^  tCS^}  or  an  A2  such  that 
(Ae  ,CS^}  =  {Az  ,CSj}.  Therefore  the  required  {{AjCS}^} 
does  not  exist,  and  therefore  the  required  Ta  does  not 
exist. 

COROLLARY  4,1:  Given  any  two  deterministic  plans, 

Wj  ,Wg,  with  tuples  {A,CS}j  e  Wj  and{A,CS}g  e  Wg, 
then  there  exists  an  abstraction  transformation  Th4 
and  plan  tuple  {A,CS}x  such  that 
f^H4  :  {{A,CS}j ,{ A, CS}g}  {{A,CS}x}  if  and  only  if 
B{A,CS}j  =  E{A  ,CS}g  andAj  —  Ag  or  CSj  =  CSg. 

Proof:  Follows  directly  from  Theorems  Hl-4. 


B.  Abstracting  Tuples  From  Two  N-deterministic 
Plans 

We  now  consider  the  plan  domain  of  N-deterministic 
plans,  Kj  and  C  Pff .  We  are  assured,  without  a  loss  of 
generality,  that  such  Kj  can  be  found  since  Pff 
N-deterministic.  We  now  examine  those  conditions  under 
which  plan  tuples  from  N-deterministic  plans  may  be 
abstracted  to  a  single  N-deterministic  plan  tuple. 

THEOREM  H5:  If  Kj  and  Kg  are  N -deterministic 
plans  with  tuples  {A, CS}j  e  Kj  and{A,CS}g  t  Kg, 
such  that  Es{A,CS}j  =  Es{A  ,CS}g  and  Aj  =  Ag  or 
CSj  =  CSg,  then  there  exists  a  plan  tuple  {A,CS}x  and 
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abstraction  transformation  T^S  such  that, 

ThS  {{A,CS}j  ,{A,CS}g}  -*{{A,CS}x}. 

Proof:  Since  the  only  difference  between  deterministic 
and  N-deterministic  plan  tuples  is  the  value  of  the  tuple 
executed  value ,  we  assert  the  following: 

1.  If  {A,CS}j  =  {A,CS}g  then,  by  Theorem  HI,  let 

{{A,CS}x}  =  {{A,CS}j}. 

2.  IfCSj  CSg  then,  by  Theorem  H2,  let 
{{A,CSW  =  {{A,CS}j}. 

3.  IfCSj  =  CS^,  then  by  Theorem  H3,  let 
{{A,CS}*;  =  {{A,CS}j}. 

Clearly,  for  each  of  the  above  state  descriptions  {{A,CS}3c}  is 
an  N-deterministic  plan  where,  rank  {{A,CSJ; }  + 
ranit{{A,CS}g}  >  rank  {{A,CS}x}  as  required.  Likewise,  by 
definition,  Es{A,CS}j  =  Es{A,CS}g  =  Es{A,CS}x  as 
required.  Therefore,  Th5  is  the  required  abstraction 
transformation. 


C.  Abstracting  Tuples  From  One  Deterministic 
Plan 

We  now  consider  the  special  case  of  abstracting 
tuples  from  a  single  deterministic  plan,  W}  .  That  is,  if 
{A,CS}j  and  {A,CS}j  -f  1  are  any  two  contiguous  plan  tuples 
belonging  to  some  deterministic  plan,  Wj  ,  then  does  there 
exist  a  deterministic  plan  tuple  {A,CS}jc  and  an  abstraction 
transformation  T>\  such  that, 

TA  :  {{A,CS}j  ,{A,CS}^}  -.{{A,CS}x}?  The  plan  tuple 
variables  being  considered  (Figure  Two)  are: 

a)  Actions  Aj  and  Aj + 1 

b)  Initial  Context  States  CSj  and  CSj  1 

c)  Final  Context  State:  CSj +2  =  E{A,CS}j-/-i. 


{CS}j 

Aj  i) 

{CS}j+, 

Aj*l 


Ta 


{CS)j 


6  Ax 


{OS}j+2  t  t  {CS}j+2 

Figure  Two:  Abstracting  a  Single  Deterministic  Plan. 

THEOREM  VI:  If  Wj  is  a  deterministic  plan  with 
contiguous  plan  tuples  { A, CS}j  and{AfiS}j  +  i  e  Wj , 
then  there  exists  a  deterministic  plan  tuple  (AyCSfx 
an  abstraction  transformation  Tvi  such  that 
TV1 :  {{AyCS}j  y{AyCS}j  +  l}  ->  {{AyCS}x}  if  and  only  if 
CSj  =  CSj-f-i . 


Proof:  The  conditions  under  which  ( A,CS}j  and  {A,CS}j  -h  i 
may  be  abstracted  by  some  Tvi  into  a  single  deterministic 
plan  tuple,  {A,CS}jc  are  those  which  satisfy  the  following 
equation, 

E(A*,CSj}  A  E{Aj-H,E{Aj,CSj}}.  (4.1) 

Since  Wj  is  a  deterministic  plan,  (3.1)  may  be  rewritten 
as,  E  (Ax, CSj}  =  E{Aj-M,E{Aj,CSj}}.  Therefore 
Ax  =  Aj+i  and  CSj  =  E{Aj ,  CSj}  =  CSj  +  2 ,  and 
{A,CS}x  =  {A,CS}j  +1.  Conversely,  CSj  =  CSj+i  = 
E{Aj,CSj}  by(4.1).  Therefore,  Ax  =  Aj^i  and 
{A,CS}x  =  {A,CS}j+i .  In  both  cases, 
ranfc  {{A,CS}x}  <  ranfr  {{A, CS}j+i}  +  rank{{AyCS}j}. 
Therefore,  Tvi  is  the  required  transformation, 

Tvi :  {{A,CS}j  ,{A,CS}j  +  i}-^  {{A,CS}x}.  q.e.d. 

Comment:  Given  some  (j  +  l)th  tuple  in  a  deterministic 
plan  Wj,  it  is  possible  to  abstract  the  jth  and  (j  +  l)th  plan 
tuples  if  and  only  if  E{j#^',  CSj }  -*  CSj.  That  is,  action  Aj, 
acting  in  context  state  CSj,  does  not  alter 
conditions/features  in  CSj,  e.g.,  the  Ax  acts  like  the 
identity  function  with  respect  to  CSj  with  the  context 
state  fixed  for  "j"  tuples.  In  this  case  the  CSx  essentially 
acts  as  a  "contextual  stage"  for  the  actor  actions. 
Examples  include  domain  independent  actions. 

Theorem  VI  can  be  applied  repeatedly  to  some 
deterministic  plan  P  with  rank  P  >  2.  That  is,  if 

P  =  {{A,CS}i,....,{A,CS}i,}, 

then  Pcan  be  recursively  abstracted  to  form  a  new 
deterministic  plan,  P’,  where, 

p-  =  {{A,CS}u} 

and  the  Ay  represents  a  composite  description  for  (plan  P) 
actions  {A^,....,  Ay}. 

D.  Abstracting  Tuples  from  One  N-deterministic  Plan 

We  now  consider  the  case  where  Kj  is  a 
N-deterministic  plan,  Kj  C  p£f. 

THEOREM  V2;  If  {AyCSjj  ,{A,CS}j-f-l  are  any  two 
contiguous  plan  tuples  belonging  to  N-deterministic 
plan  Kj,  such  that  Bf{A,CS}j  for  f  >  1  and 
Eh{A,CS}j^l  fork  >  1,  then  there  exists  no  abstraction 
transformation  TV2  and  N-deterministic  plan  tuple 
{A,CS}x  such  that, 

Tvi:  {{A,CS}j  ,{A,CS}j  +  i}  ^{{A,CS}xh 

Proof:  As  in  Theorem  VI,  we  consider  those  conditions 
such  that  equation  (4.1)  is  true  (Figure  Three).  In 
particular. 


CSj  :!=  Ef{A,CS}j 

is  never  tvxxe  since,  CSj  ^  {CSg, CSq  -hf-l}  ”  Ef{A,CS}y 
where  by  definition  f  >  1  since  {A,CS}j  an  N-deterministic 
plan  tuple  in  Kj .  q.e.d. 
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VI.  PLAN  ABSTRACTION  ERRORS 


{CS)j 


'Ms 


{CSq, CSq+^,i} 
^j+1 


{CS}J 


//K 

{CSa, CSa+ft-i}  {CSa, ....  CSa+A-/} 


Figure  Three:  Abstracting  a  single  N-deterministic 
Plan. 


A  simple  corollary  to  Theorem  V2,  which  is  not 
proved,  is, 

COROLLARY  V2a:  If  {A,CS}j,  {A,CS}j+i  are  any  two 
contiguous  plan  tuples  belonging  to  N-deterministic  plan 
Kj,  such  that  rank  £{A,CS}j  =  l,then  there  exists  an 
abstraction  transformation  T\/2a  and  N-deterministic 
plan  tuple  { A, CS}x  such  that, 

TV2a  :  {{A,CS}j  dA,CS}j^i}  ->  {{A,CShl 


V.  PLAN  ABSTRACTION  TRANSFORMATION 
SUMMARY 


We  have  shown  that  deterministic  plans  may  be 
accurately  abstracted  using  the  following  transformations: 
Tvi  (see  Theorem  VI),  and  Th4  (see  Corollary  4.1),  as 
applicable.  Both  of  these  abstraction  transformations, 
however,  require  extensive  a  priori  information  concerning 
the  plan’s, 

Al.  Context  State,  when  using  transformation  T]/i,  or, 

Bl.  Either  Context  State  or  Actor  Action,  when  using 
transformations  Th4. 


The  TASC  TSA  research  effort  has  been  directed, 
under  U.S.  Government  contract,  to  develop  a  domain 
independent  and  extensible  TSA  paradigm.  TSA  depends 
upon  the  system's  ability  to  recognize  ongoing  or 
developing  enemy  tactical  plans.  The  TASC  TSA 
automated  plan  recognizer  paradigm  is  the  problem  solver 
which  attempts  to  recognize  emerging  or  ongoing  tactical 
events  by  comparing  intelligence  sensor  data  with  a  set  of 
instantiated  tactical  scripts/plans.  Incoming  sensor  and 
terrain  data  are  first  analyzed  by  the  TSA  P/an  Design 
Module  (PDM).  Using  the  terrain  information  and  a 
knowledge  of  the  principles  of  warfare,  the  PDM  produces 
output  that  helps  parameterize  the  skeleton  tactical  scripts 
which  are  contained  in  a  script  library  (scripts  written  by 
The  Analytic  Sciences  Corporation  HERO  group  of 
military  experts  and  historians).  The  purpose  of  the  Plan 
Evaluation  Module  (PEM)  is  to  associate  the 
preconditioned  tactical  scripts  with  processed  intelligence 
data.  Temporal  and  causal  relationships  between  tactical 
scripts  are  represented  using  a  somewhat  modified  [Allen, 
1983,  1985]  version  of  Wilensky's  KODIAK  plan  relation 
language  [Wilensky,  1984].  Scripts  are  used  in  KODIAK 
to  represent  highly  stereotypical  tactical  information 
which  corresponds  to  generally  accepted  practices  for 
conducting  warfare. 

KODIAK  is  a  relation-based  representation  language 
where  the  objects  of  the  relation(s)  are  absolutes, 
relations  and  aspectuals  [Hatfield],  Absolutes  can 
represent  physical  objects  (AC,  or  CSi),  actions  (Aj), 
events  (CSj)  or  abstract  ideas  (Gp).  Relations  capture 
relations  between  absolutes  and  aspectuals  are  the 
arguments  of  the  relation.  Temporal  and  causal 
relationships  are  captured  using  associa/ions  between 
Objects.  Figure  Four  shows  a  KODIAK  representation  of 
a  Maneuver  Plan  [Hatfield].  A  KODIAK  representation  of 
an  enemy  tactic  resembles  an  augmented  N-deterministic 
plan  of  tactical  scripts.  The  plan  is  augmented  in  the 
sense  that  both  causal  and  temporal  relations  between 
objects  are  specified.  KODIAK  scripts  may  also  be 
represented  as  plans.  We  will  now  prove  that  if  these 
tactical  scripts  are  improperly  abstracted,  then  the  PEM 
will  not  function  properly. 


A.  KODIAK  Action  Scripts 


In  particular,  T\/t  may  be  employed  for  plan  actions  which 
are  insensitive  to  elements/features  found  within  the 
actor's  contextual  domain,  CS. 

N-deterministic  plans,  however,  are  only  abstractable 
whenever, 

A2.  A  neighboring  plan  tuple  appears  deterministic, 
or, 

B2.  Tuple  Execution  values  are  equal,  and  known  a  priori. 

If  case  A2  above  is  true,  transformation  T\/2a  may  be 
used.  Th5  may  be  employed  when  case  B2  conditions  are 
true.  In  general,  N-deterministic  plans  which  contain  few 
deterministic  plan  tuple  elements  are  not  good  candidates 
for  plan  abstraction. 


Each  KODIAK  Action  generally  contains  several 
slots  which  represent  action  preconditions  and 
postconditions,  KODIAK  actions  may  also  be  represented 
as  a  plan  where  action  names  represent  plan  names  or 
S3mibols,  action  preconditions  represent  features  in  CSi 
believed  to  be  necessary  for  the  plan  to  be  executed,  and 
action postcondition( s)  represent  plan  goals  or  goal  context 
states.  For  example,  the  KODIAK  Action  "red 
breakthrough  at  loc  1"  is  the  name  of  a  plan  whose  goal  is 
"red  controls  loc  1".  The  important  contextual  features  to 
check  which  make  the  execution  of  plan  Red  Breakthrough 
at  loc  1  possible  are  given  as  action  preconditions  "Blue 
Gap  at  loc  1"  and  "Red  Force  at  loc  1"  (Figure  Four).  That 
is. 
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Red  Breakthrough  at  tod  = 

{{A,CS}2,...,  {A,CS}fe }  ^  Red  Controls  loc  1  (6-1) 

where  blue  gap  at  loc  1  and  red  force  at  loc  t  are 
states/conditions/features  which  must  be  present  within 
CSi. 


-|  Event  I 


f  D 

°  T 

♦ 

Blue 
surface 
at  loc  2 

Blue 

gap 

at  loc  1 

/ 

Red 
force 
at  loc  1 

Red 

controls 
loc  1 

Blue  force  diverting 
from  loc  to  loc  2  at  t| 


Observation 


Prediction 


Figure  Four:  Representation  of  Frontal  Assault  With 
Massed  and  Refused  Flanks. 


Other  than  action  preconditions  and  postconditions, 
no  other  checks  (conditions! states)  are  used  to  check  the 
(ongoing)  status  of  the  KODIAK  action  (see  Section  B  and 
Named  Areas  of  Interest).  We  now  prove  that  this 
representation  of  tactical  script-based  action  is  only 
correct  when  the  (action)  plan  is  either  deterministic  or 
can  be  represented  as  the  union  of  a  finite  number  of  very 
simple  deterministic  plans. 

THEOREM  6-1:  Let  P  be  a  plan  with  a  priori  specified 
context  CSi  and  plan  goal  context  CSn-hl  such  that 
GpzCSn-i-l=^{^>^^}n- 

IfF'={{A,CS}i:  >Ii,l  <i  <  n,where{A,CS}i  z  P  and 
AC(P)  =Gph  lohereP*^  0,  then  P*  is  a  deterministic 
plan. 

Proof:  Assume  P  =  { {A,CS}j  {A,CS}7i}  with 

f{A.CS}n  -►CSa+ias  specified.  P  canbe  written  as  P={ 
{A,CS}i ,  P {A,CS}n}  where  P  'C  P  and  P'  a  plan.  Now 
assume  for  any  tuple  {A,CS}g  e  P', 

£{Ag,CSe}  =  f  {Ag,CSx}  for  anyCSjc,  in  particular  for 
CSg  CSx.  Show  that  P  'deterministic. 

Since  f{Ae,CSe}  =  E  {Ag,CSx},  then 
£{Ag,  CSg}  =  E  {Ae,  E  {A ,  CS}g-2 }  for  all  {A ,  CS};i, 
h=l,...,(e-l)-  In  particular,  then 
E{Ae,  CSg}  =  E  {Ae,  £  {A,CS}j  }  for  {Ag.CSe}  the  last  tuple 
in  P Then  P  =  {{A,CS}2 ,  {A,CS}e,  {A,CS};i}  where 
P'=  {{A,CS}e}.  If  tuple  {A,CS}i  deterministic,  then 
{{A,CS}i ,  {A,CS}e}  deterministic,  and  therefore,  P ' 
deterministic,  as  required.  Assume  {A,CS}i 
N-deterministic,  where  £{Ae,CS2}  =  {CSi,...,CSp},  Since 
£{Ae,CSi}  =  £{Ae,CSp}  for  every  element  of  £{Ae,CSi}, 
and  Ag  equal  for  all  such  tuples,  by  Corollary  4.1  these 
{Ag,  CSj}  can  be  abstracted  to  a  single  tuple.  That  is, 
{{A,CS}2,  {A,CS}g}  may  be  written  as  some  {{Ag,CSi}}. 
Therefore,  P '  deterministic,  as  required,  q.e.d. 


COROLLARY  6.1;  If  plan  P  as  in  Theorem  6-1,  then 
either  P  deterministic,  or  there  exists  n  deterministic 
plans  Wj  such  that, 

P  =  U  Wj. 

j=l 

Each  Wj ,  if  it  exists,  has  a  very  simple  deterministic  form 
equal  to  {{Ag,CS2},/A,CS72}  where,  Pi{A,CS}z  —  {CSn.-f2} 
as  required. 


B.  Plan  Projection/Evaluation  Errors  Resulting 
from  Improperly  Represented  Action  Plans 

If  a  KODIAK  action  script  doesn’t  have  a  plan  form 
as  described  in  Corollary  6.1,  then,  at  a  minimum,  subtle 
errors  will  occur  within  the  PEM  process.  If  the  action 
plan  form  is  incorrect,  the  PeF  form  of  the  plan  can  be 
quite  large  and  complicated  with  numerous  (unforeseen) 
action/contextual  state  interactions  with  other  features 
within  the  tactical  domain,  e.g.,  other  forces,  environment. 
The  KODLA.K  methodology  of  only  checking  prescribed 
action  preconditions  and  post  conditions  will  be 
insufficient  to  accurately  monitor  the  script  action.  The 
action  script,  improperly  represented,  will  not  represent 
stereotypical  but  rather  domain  dependent  actionis),  and 
the  subsequent  execution  of  the  action  will  depend  heavily 
upon  prevailing  domain  context.  Fortunately,  these  types 
of  PEM  errors  can  be  eliminated  or  significantly  reduced 
by  making  use  of  any/all  of  the  following 
recommendations. 

1 .  Abstract  the  script  properly  into  a  plan  form  as 
described  in  Corollary  6.1. 

2.  Rewrite  the  action  script  into  a  deterministic  form,  as 
described  in  Corollary  6.1 . 

3.  If  the  script  can 't  be  rewritten  into  a  deterministic  form, 
but  is  truely  N-deterministic,  then  either, 

a.  rewrite  it  in  a  form  resembling  equation  Lemma  3.1 , 
part  3,  noting  each  CSi  and  check  for  each  CSi  as 
part  of  the  KODIAK  pre-postcondition  check,  or, 

b.  rewrite  the  KODIAK  representation  (e.g.,  Figure 
Four)  into  a  form  which  makes  it  possible  to 
implement  1,2,  or  3  above. 

It  has  been  recently  brought  to  the  attention  of  the 
author  that  TASC  had  forseen  this  type  of  problem  and  had 
discussed  it,  albeit  briefly,  as  an  implementation  issue 
[Hatfield,  p.  12-14]  called  Named  Areas  of  Interest  (NAI). 
The  NAI  are  computed  features  (CSj)  which  are  checked  to 
monitor  the  progress  of  ongoing  actions.  The  NAI  are 
similar  to  recommendation  3a  above.  NAI’s  are  computed 
using  terrain  features  (see  Create  Plan  Network 
Submodule).  Mr.  Stephen  Williams,  principal  investigator 
for  the  TASC  TSA  paradigm  implementation  effort,  has 
spoken  with  the  author  concerning  a  similar  problem. 
TASC  is  attempting  to  develop  criteria  to  evaluate 
appropriate  movement  (actions)  which  appear  to  be 
suboptimal  locally  but  which,  nevertheless,  successfully 
contribute  to  an  optimal  global  solution.  Work  is  presently 
underway  between  TASC  and  the  author  to  incorporate 
some  of  the  theoretics  developed  within  this  paper  into  the 
PEM. 
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C.  A  Situation  Analysis  Paradox 

A  situation  assessment  analysis  paradox  can  arise 
when  using  the  TASC  proposed  TSA  paradigm  with 
incorrectly  abstracted  KODIAK  action  scripts.  The  PEM 
will  incorrectly  assume  that  the  action  scripts  have  a 
deterministic  plan  form.  As  such,  the  PEM  has  been 
programmed  to  make  a  closed  set  of  assumptions  about 
plan  actions  which  are  no  longer  necessarily  true'  For 
example,  assume  there  exists  some  KODIAK  action  plan  P 
with  precondition  Pp/?f  and  post  condition  Pposr  which 
have  been  defined  a  priori,  as  described  in  section  Vl-b. 
Once  state  Pposr  is  detected  by  PEM,  the  action  Pis 
assumed  to  be  completed.  This  conclusion,  however,  is  only 
true  when  P  has  the  correct  plan  form  (Corollary  6.1)  and 
when  both  Pppf  and  Pposrhave  been  reliably  detected.  If 
P  has  been  incorrectly  abstracted,  for  example,  then  the 
presence  of  state  PpoST  niay  have  nothing  to  do  with  either 
the  ongoing  or  completed  state  of  action  P.  Given  this  kind 
of  PEM  performance,  a  TSA  analysis  paradox  can  arise  as 
follows. 


Cl.  Uniform  Probability  Distribution  Assumption 

Assume  that  for  some  action  P  that  the  likelihood 
that  state  Pposris  both  present  in  the  domain  and 
accurately  detected  by  the  PEM  is  best  described  as  a 
random  process,  i.e.,  uniform  probability  distribution. 
Then,  the  likelihood  that  a  plan  recognizer  could  accurately 
detect  the  presence  of  some  action,  P/,  and  discriminate  Pj 
from  some  other  action,  Pj ,  is  problematic.  A  situation 
assessment  system,  relying  upon  accurate,  and  robust, 
plan  detection  will  tend  to  focus  on  whatever  tactical 
scenario  can  be  accounted  for  with  the  least  amount  of 
error.  Unfortunately,  the  plan  recognition  mechanism  is 
not  only  used  to  accurately  detect  plans,  but  is  also  used  to 
prune  plan  scenarios  from  further  consideration.  A  TSA 
system  which  is  incapable  of  accurate  low-level  plan  action 
discrimination  will  eventually  consider  a  broad  base  of 
possible  tactical  hypotheses.  The  system  will  attempt  to 
focus  its  attention  on  that  tactical  script  which  appears 
"best".  Discernment  of  a  "best"  hypothesis  involves  a 
prescription-based  validation  process  where  the  presence 
or  absence  of  domain  features  and  attributes  is  used  to 
establish  the  belief  in  an  ongoing  tactical  hypothesis.  At 
this  point,  one  of  two  possible  conclusions  will  be 
developed.  If  the  low-level  discrimination  process  involves 
monitoring  tactical  KODIAK  actions,  then  the  system  will 
randomly  find  evidence  for/against  every  possibility.  If 
the  low-level  discrimination  process  does  not  rely  upon 
evidence  accrued  using  KODIAK  actions,  then  the  system 
will  evolve  through  a  (possibly  lengthy)  process  of 
hypothesis  testing  where  the  evidence  for  previously 
considered  hypotheses  will  tend  to  accumulate  over  time. 
In  either  case,  the  system  will  eventually  arrive  at  a 
position  where  there  exists  sufficient  evidence  to  posit  the 
presence  of  a  large  number  of  competing  tactical 
scenarios,  but  with  no  "clearly  best"  tactical  scenario  as  yet 
targeted.  Faced  with  this  body  of  contending  tactical 
hypotheses,  each  accumulating  evidence  for  its  existence 
in  time  where  each  hypothesis  is  considered  equally  likely, 
the  system  will  be  forced  to  make  use  of  a  priori  tactical 
threat  information. 


D.  Effect  of  PEM  Errors  Upon  Tactical  Command 
and  Control 

When  faced  with  a  variety  of  possible  threats,  where 
each  is  equally  likely  to  represent  an  opposing  enemy 
tactic,  the  most  conservative  rationale  is  to  focus  attention 
on  those  tactical  indicators  which  reliably 
establish/discard  the  presence  of  enemy  tactics  which  pose 
the  greatest  threat  to  friendly  forces.  This  is  a  type  of 
"worst  case  scenario"  logic  which  is  used  when  some  enemy 
action  appears  imminent  but  unknown.  If  one  is  unable  to 
decide  what  an  adversary  is  planning,  but  strongly 
suspects  that  some  adversarial  plan  is  ongoing,  then  it 
becomes  imperative  to  thoroughly  check  for  the  presence  of 
those  adversarial  plan(s)  which  can  potentially  inflict  the 
most  damage.  If  sufficiently  motivated,  on  the  basis  of 
assumed  threat,  to  accurately  detect  the  presence  of  some 
plan  indicator(s),  the  PEM  as  described  herein  will 
eventually  find  evidence  for  such.  The  fact  that  the  threat 
evidence  will  not  be  consistently  detected  is  easily 
rationalized  in  terms  of  the  importance  of  this 
information.  Information  which  is  most  critical  is  apt  to  be 
that  which  is  most  difficult  to  detect.  This  type  of  TSA 
logic  will  inevitably  result  in  a  system  performance 
converging  to  one  of  two  modes:  either  1)  assessing  the 
tactical  domain  as  essentially  undecipherable,  where  every 
possible  enemy  tactic  is  seen  as  equally  likely,  or,  2)  adopt 
a  "worst  case"  scenario  where  the  Blue  Force  Commander's 
worst  fears  are  realized.  Both  situation  assessments  are, 
of  course,  in  error.  Worse  than  that,  however,  the  former 
can  hide  or  mask  real  danger,  while  the  latter  can  provoke 
a  dangerous  Blue  Force  overreaction  which  can  result  in  a 
rapid  escalation  of  the  tactical  situation. 


VII.  CONCLUSIONS 


A  mathematical  basis  for  plan  abstraction  (reduction) 
has  been  developed.  We  have  shown  that  the  plan 
recognition  process,  as  evidenced  by  the  KODIAK  use  of 
tactical,  script-based  actions,  will  be  inaccurate  if  the 
action  scripts  are  not  written  in  a  plan  form  as  described  in 
Corollary  6.1  of  this  paper.  Some  evidential  and 
experimental  evidence  for  these  theoretical  predictions  has 
been  reported.  Recommendations  have  been  made  to 
remove  or  minimize  these  errors.  However,  at  least  one 
recommendation  would  possibly  require  frequent 
contextual  checks  of  ongoing  action  execution.  These 
additional  processing  requirements  may  make  an 
automated  TSA  process  less  attractive  to  the  user. 

We  concluded  by  noting  that  plan  recognition  errors 
are  particularly  insidious  since  they  may  mask  other 
system  errors.  The  result  can  be  a  type  of  analysis  paradox 
where  the  system  is  unable  to  distinguish  hypothetical 
enemy  plans  from  factual  ones.  Commanders  will  find  it 
difficult  or  even  dangerous  to  make  command  decisions 
since  (given  sufficient  time)  the  TSA  may  converge  to  an 
enemy  "worst-case"  scenario. 
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INTRODUCTION 

ESAU  (Expert  System  AOB  Update)  is  an 
expert  system  command  and  control  decision  aid, 
currently  under  development  for  RADC/COAD.  ESAU 
is  intended  to  assist  intelligence  personnel  in  the 
Tactical  Air  Command  Center  (TACC)  with  the  task 
of  keeping  track  of  the  counts,  locations  and 
capabilities  of  enemy  aircraft.  This  information  is 
maintained  in  the  Air  Order  of  Battle  (AOB). 

An  essential  role  In  management  of  a  modern 
air  battle  is  keeping  track  of  the  enemy  aircraft  - 
where  they  are  based,  what  weapons  and  support 
facilities  are  available  to  them,  what  sorties  are 
being  mounted  from  what  airfields,  and  so  on.  This 
information,  together  with  understanding  of  Soviet 
tactics  and  doctrine,  enables  the  intelligence  staff 
to  recognize  tactically  significant  developments  in 
the  air  battle,  and  thereby  to  support  the 
commander's  ability  to  respond  appropriately. 

The  air  order  of  battle  consists  of  a  list  of 
enemy  military  aircraft,  including  a  count  of 
aircraft  by  type,  model,  and  country  code  for  each 
airfield  in  the  theater.  The  AOB  is  maintained  by 
an  AOB  analyst,  on  the  basis  of  a  number  of 
different  kinds  of  intelligence  reports.  The  AOB 
analyst's  job  is  to  maintain  the  AOB  in  as  complete 
and  correct  a  condition  as  possible.  In  a  rapidly 
changing  wartime  environment.  This  is  a  data 
fusion  process,  requiring  judgment  and 
understanding,  because  the  reports  are  often 
incomplete,  uncertain,  or  even  contradictory. 

Keeping  track  of  this  information  involves 
considerable  amounts  of  what  is  called  'bean 
counting’:  counting,  sorting,  transcribing 

information  from  messages,  etc.  This  bean 
counting  is  necessary  to  support  the  analysis, 
which  leads  to  the  valuable  resulting  information. 
Bean  counting  to  some  extent,  and  analysis 
especially,  are  largely  data  fusion  -  combining 


information  from  a  number  of  sources,  extracting 
significance  and  meaning  from  large  and  diverse 
amounts  of  detail. 

In  current  practice,  these  fusion  processes 
are  accomplished  entirely  manually.  Many  of  the 
bean  counting  tasks  are  especially  appropriate  for 
automation;  further,  the  tasks  that  involve  some 
element  of  judgment,  knowledge  of  enemy  doctrine, 
etc.,  can  be  materially  assisted  by  decision  aids 
based  on  artificial  intelligence  technology. 

ESAU  will  support  the  AOB  analyst  by 
assisting  with  many  of  the  bean  counting  tasks,  and 
by  using  expert  system  techniques  to  generate 
recommendations  of  updates  to  the  AOB,  based  on 
events  reported  In  incoming  intelligence  reports. 

In  this  paper  we  describe  ESAU  and  the  AOB 
analysis  process  in  terms  of  the  C2  system  of 
which  it  is  a  part,  the  kinds  of  intelligence  inputs 
which  it  uses,  the  specific  tasks  performed  by  the 
analysts,  and  the  role  of  ESAU  in  supporting  the 
analyst. 


BACKGROUND 

ESAU  is  the  next  generation  of  the  DAGR 
(Dynamic  Order-of-Battle  Aggregation  Aid) 
program,  also  developed  by  RADC/COAD.  DAGR  was 
a  proof  of  concept  system  that  applied  expert 
system  technology  to  the  problem  of  assisting  an 
analyst  to  maintain  an  up  to  date  Air  Order  of 
Battle  estimate,  based  on  incoming  air  track 
information  and  engagement  reports. 

In  its  formal  evaluation,  DAGR  was  found  to 
enable  untrained  personnel  to  perform  the  AOB 
analysis  role  more  effectively  than  trained 
analysts  were  able  to  without  use  of  the  aid.  DAGR 
did  much  of  the  bean  counting,  and  called  attention 
of  the  user  to  appropriate  changes  to  the  AOB,  so 
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that  untrained  personnel  were  able  to  do  an 
effective  job.  DAGR  also  Improved  the 
effectiveness  of  trained  personnel. 

ESAU,  currently  under  development,  will  be  a 
free  standing  microprocessor-based  aid.  It  will 
use  real  message  traffic,  and  will  provide  a  robust 
capability  to  assist  in  maintaining  the  AOB  in 
realistic  exercise  and  operational  environments. 
ESAU  will  perform  data  fusion  functions,  including 
correlation  of  air  track  data  and  engagement 
reports,  and  assisting  in  recognition  of  tactically 
significant  developments  in  the  air  battle. 

The  officers  charged  with  maintaining  the 
AOB  are  in  the  Intelligence  section  of  the  Tactical 
Air  Command  Center  (TACC).  Thus  ESAU  will 
provide  assistance  with  an  existing  functional  role 
in  the  context  of  the  overall  TACC.  The  structure 
and  organization  of  the  TACC  is  well  established, 
as  is  the  role  of  the  AOB  analyst  within  it.  ESAU's 
development  plan  avoids  the  usual  technologist's 
temptation  to  try  to  redesign  the  whole  C2  system. 
Rather,  ESAU  addresses  specific  functions  which 
can  be  improved  with  automation,  leaving  the 
overall  flow  essentially  intact  but  more  effective. 


E^AU  IN  THE  TACTICAL  AIR  COMMAND  CENTER 

To  understand  the  role  of  ESAU,  consider  the 
overall  structure  and  function  of  the  TACC.  The 
TACC  is  divided  into  two  main  sections:  Current 
Operations  and  Current  Plans.  The  intelligence 
section  of  Current  Ops  is  ENSCE;  Enemy  Situation 
and  Correlation  Element;  in  Current  Plans,  the 
intelligence  section  is  CID:  Combat  Intelligence 
Division. 

Current  Ops  conducts  today's  war;  this 
includes  conducting  air  defense,  responding  to 
problems  and  opportunities  as  they  arise  in  the  air 
battle,  and  executing  the  planned  air  tasks.  The 

plan  is  contained  In  the  daily  Air  Tasking  Order 
(ATO),  and  supplementary  air  tasking  messages. 

The  ATO  is  a  comprehensive  order  assigning 
aircraft  and  weapons  from  specific  units  to 
targets,  with  compositions  of  attack  packages, 
time  on  target  (TOT),  tanker  and  EC  support,  etc. 
The  ATO  is  produced  on  a  daily  basis  in  the  Current 
Plans  section  of  the  TACC.  In  addition  to  producing 
the  ATO,  Current  Plans  is  responsible  for  assessing 
enemy  capabilities,  and  various  orders  of  battle 
(OBs),  including  missile  (MOB),  electronic  (EOB)  and 
air  (AOB).  This  capabilities  assessment,  together 
with  an  assessment  of  the  enemy's  probable  course 
of  action  (PCA),  is  used  to  support  the  commander's 
staff  situation  briefing,  to  provide  information  on 
an  Informal  basis  to  other  TACC  staff,  and  to 
support  the  planning  of  the  next  day's  ATO. 

The  TACC  operates  on  messages  -  some  voice 
communication,  some  handwritten  hardcopy,  but 
much  of  It  in  electronic  digital  form.  These 


electronic  messages  are  typically  well  formatted, 
and  much  of  the  Important  information  is  indeed 
contained  in  easily  decoded  fields.  Significant 
information  is  also  contained  in  free  text  fields 
such  as  AMPN  (amplification). 

Within  a  TACC,  information  Is  stored  and 
presented  mainly  on  wall  maps,  with  some 
summaries  and  tabulations  being  hand  generated  in 
hardcopy.  These  wall  maps  are  an  essential 
element  of  the  TACC,  because  they  enable  groups  of 
people  to  discuss  the  situation  in  a  way  that  small 
computer  generated  displays  can  not.  There  are 
wall  maps  for  each  major  function:  planned 
reconnaissance  missions,  enemy  ground  forces, 
enemy  air  defenses,  etc.  In  particular,  there  are 
AOB  maps,  typically  one  showing  airbases  with 
aircraft  assigned,  the  other  showing  plots  of 
sorties  and  estimated  enemy  sortie  corridors. 


TACC  operations  at  present  are  essentially 
manual.  There  are  electronic  message  handling  and. 
management  systems,  and  the  TACC  personnel 
themselves  have  produced  some  small  application 
tools  that  run  on  the  Air  Force  Z150s  and  Z248s. 
But  current  practice  does  not  include  any 
substantial  automated  support  for  understanding 
the  meaning  and  importance  of  the  messages, 
maintaining  the  OB  databases,  generating 
estimates  and  briefing  materials,  or  any  of  the 
other  myriad  interpretation  and  bean  counting 
tasks  of  the  TACC  staff. 

The  TACC  is  a  large  and  complex  organization, 
with  many  well-specified  technical  roles,  of  which 
AOB  analysis  is  just  one.  For  the  remainder  of  this 
paper,  we  will  concentrate  only  on  the  AOB  role. 


TASKS  PERFORMED  BY  AOB  ANALYSTS 

ESAU  is  a  decision  aid  for  the  AOB  analyst. 
To  understand  how  ESAU  can  help  the  analyst, 
consider  the  nature  of  the  analyst's  job.  The  role 
of  the  AOB  analyst  is  to  merge  the  incoming 
information  in  messages  with  his  knowledge  of 
Soviet  tactics  and  methods,  and  with  preexisting 
reference  data,  to  develop  an  up  to  date 
understanding  of  the  enemy's  forces,  capabilities, 
and  intentions.  To  do  this,  he  performs  a  number  of 
specific  tasks,  as  follows. 

AOB  analysts  tabulate  enemy  sorties  by  day, 
by  mission  type,  by  base  of  origin,  and  possibly  by 
other  characteristics  as  desired.  Enemy  routes  of 
flight  are  plotted  on  a  wall  map,  in  order  to 
recognize  sortie  corridors. 

The  AOB  analyst  attempts  to  recognize 
tactically  significant  events,  such  as 
redeployments,  or  sequences  of  events  which  are  to 
be  expected  based  on  Soviet  doctrine.  For  example, 
deployment  of  the  Mainstay  (the  Soviet  equivalent 


of  our  AWACCS)  aircraft  may  be  typically  followed 
by  more  localized  jamming,  and  then  by  a  ground 
attack.  Soviet  doctrine  is  explicit  and  rigid,  so 
that  a  prime  objective  is  to  anticipate  their  moves, 
deny  their  critical  steps,  and  thereby  reduce  their 
effectiveness.  Understanding  of  these  sequences 
and  of  enemy  corridors  can  lead  to  better  air 
defense  decisions  as  well  as  recognition  of  our 
vulnerabilities,  enemy  PCA,  and  targeting 
opportunities. 

For  example,  In  one  exercise  it  was  strongly 
desired  to  destroy  or  prevent  the  operation  of  a 
Mainstay  aircraft.  The  Mainstay  was  defended  by 
Foxbats,  sufficiently  so  that  our  air  to  air  missions 
were  ineffective  against  it.  One  of  the  AOB 
officers  observed  that  the  Foxbats  were  based  at  a 
relatively  undefended  field,  so  that  by  attacking 
the  Foxbats  on  the  ground  we  could  defeat  the 
Mainstay.  This  plan  conserved  valuable  air-to-air 
assets,  and  neutralized  the  Mainstay  with  a 

relatively  economical  bombing  raid.  This  is  an 
example  of  a  targeting  opportunity  being 

recognized  by  an  AOB  officer. 

Redeployments  of  aircraft  from  one  base  to 
another  can  be  significant  events:  the  aircraft 

themselves  are  the  primary  threats,  and  the 
primary  targets.  The  base  facilities  only  support 


them.  Deployment  of  new  aircraft  into  the  theater, 
or  the  positioning  of  new  capabilities  such  as  CBW 
capable  systems,  are  important  to  recognize. 

Again,  there  are  some  predictable  signs  of 
impending  redeployments  -  pre-positioning  of  fuel, 
weapons,  and  support  facilities,  for  example,  and 

recovery  of  aircraft  at  other  than  their  base  of 

origin.  The  association  of  killed  aircraft  with 
their  base  of  origin  is  one  of  the  main  tasks  of  the 
AOB  analyst,  and  provides  the  basis  for  accurate 
estimates  of  enemy  dispositions. 

In  summary,  the  AOB's  task  is  to  perceive  and 
understand  enemy  activity  related  to  air  strength. 

This  is  a  matter  of  judgment,  training  and 

experience,  and  requires  the  analyst  to  be  able  to 
think  through  the  evidence  and  discuss  it  with 
other  members  of  the  TACC  staff.  In  support  of 
this  judgmental  responsibility  there  are  a  number 
of  bean  counting  tasks  which  are  currently 
executed  manually,  and  which  take  up  most  of  the 
AOB  analyst's  time. 


SUMMARY  OF  THE  AOB  RELATED  MESSAGES 

Figure  1  shows  the  flow  of  AOB  related 
information  in  the  TACC.  There  are  three  primary 


Radar,  national 
resources  etc. 


Sensitive  Air 
Track  Info 


Direct  Support  Unit  (DSU) 

Inteipret  air  track  info. 
Assign  track  numbers. 
Create  TACREPs. 


ENSCE  AOB  cell  (in  Current  Ops,  today's  war) 

Collect  AOB  related  TACREPs. 

Plot  active  AC  tracks. 

Alert  Air  Defence  as  needed. 

Log  the  TACREPs. 

Send  TACREPs  to  CID  AOB 


Results  of  Engagements: 

KILLSUM 

Results  of  Offensive  Counter-Air,  Reconnaissance; 
INFLTREP  (pilot  reports  inflight) 

MISREP  (free  text,  post  flight  debrief) 
ffIR  (initial  photo  Intel) 

RECCEXREP  (detailed  photo  interp) 

Evaluations  from  elsewhere: 

INTREP  (free  text  Intel  interpretation) 

BDA  (bomb  damage  assessment) 

SrrSUM  (situation  summary) 


TACREPs 


CID  AOB  Cell  (in  Current 
Typical  Bean  Counting  Tasks 


Plans  -  tomorrow's  war) 

Typical  Expert  AOB  Analysis  Tasks 


Identify  and  correct  errors  in  TACREPs  as  possible. 

Sort  TACREPs  by  mission  type,  AC  type. 

Tabulate  enemy  sorties  by  mission  type,  day,  AC  type,  base 
of  origin,  target. 

Transcribe  data  for  briefings,  messages 
Plot  routes  of  flight 


Associate  KILLSUM  aircraft  with  TACREP  tracks. 

Associate  killed  AC  with  base  of  origin. 

Recognize  redeployments. 

Identify  attack  corridors. 

Estimate  enemy  strength  at  each  airbase  by  initial  strength 
losses,  additions,  net  strength. 

Evaluate  threat:  likely  targets,  applicable  enemy  resources, 
enemy  preparations  and  initial  actions  in  known 
sequences. 

Identify  targets:  high  threat  airbases,  navaids,  lightly 
defended  escorts,  critical  resources. 


Figure  1:  AOB  Related  Information  Flow  within  the  TACC 
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kinds  of  information  about  aircraft:  air  track 
information,  results  of  engagements  of  enemy 
aircraft,  and  results  of  offensive  counter-air  (OCA) 
missions,  which  target  aircraft  and  facilities  on 
the  ground. 

Air  track  information  is  based  mainly  on 
radar  and  other  tracking  systems,  and  comes  in 
from  a  specialized  intelligence  unit  called  the 
Distributed  Support  Unit  (DSU).  The  DSU  receives 
inputs  from  highly  classified  intelligence  sources 
and  national  resources.  The  DSU  processes  the 
incoming  information,  manually  sanitizes  it  to  the 
SECRET  level,  and  hand  types  it  Into  messages 
called  TACREPS  (Tactical  Reports).  TACREPS 
contain  air  track  information;  their  track 
numbering,  and  therefore  the  correlation  of 
multiple  observations  with  tracks,  is  highly 
reliable.  TACREPS  provide  information  such  as 
time  of  observation,  aircraft  type  and  model, 
speed,  heading,  altitude,  etc. 

The  second  major  kind  of  information  about 
aircraft  airborne  is  engagement  reports.  When 
either  aircraft  or  ground  defenses  engage  enemy 
aircraft,  they  report  the  results,  including  time, 
location,  type  and  model,  type  of  engagement,  etc. 


Engaged  forces  have  no  way  of  knowing  about 
the  information  in  the  TACREPS;  they  merely 
report  what  they  know  from  their  engagement. 
Thus  it  falls  to  the  AOB  analyst  to  combine  the 
information;  in  particular,  one  of  the  major 
responsibilities  is  to  determine  the  base  of  origin 
for  killed  or  landed  aircraft.  This  is  essential  in 
order  to  know  which  base  to  decrement,  and 
therefore  what  change  has  occurred  in  the  enemy 
AOB.  This  information  is  also  basic  to  recognition 
of  redeployments  and  estimation  of  sortie 
generation  capability. 

Information  about  aircraft  on  the  ground 
comes  from  flight  crews  of  offensive  counter-air 
(OCA)  missions,  from  reconnaissance  missions,  and 
from  national  resources.  This  information  relates 
to  aircraft  and  facilities  observed  as  damaged, 
destroyed,  or  otherwise  changed. 

These  three  kinds  of  information  -  track  data, 
engagement  results,  and  OCA  results,  are  the  main 
inputs  for  the  data  fusion  process  that  Is  done  by 
the  AOB  analyst.  These  three  kinds  of  information 
are  conveyed  in  the  TACREPS,  KILLSUMS,  and 
reconnaissance  reports,  respectively.  In  the 
following  sections  we  describe  these  messages  and 
how  they  are  used  by  AOB  analysts  and  ESAU. 


A  DAY  IN  THE  LIFE  OF  A  TACREP 

TACREPS  are  produced  in  the  highly  classified 
DSU.  They  contain  air  track  information  from 
sensitive  sources,  manually  sanitized  and 
distributed  at  the  collateral  level.  They  are  sent 


as  messages  to  the  Current  Operations  section  of 
the  TACC  for  immediate  action.  In  particular,  they 
are  sent  to  the  ENSCE  Air  TACREP  desk,  at  teletype 
rate,  30  characters/sec.  There  the  track's  latest 
position,  heading,  speed  and  altitude  are  plotted 
and  air  defense  is  alerted  as  appropriate.  Then  the 
messages  are  logged.  Collected  messages  are 

forwarded  periodically  to  the  Current  Plans  section 
of  the  TACC,  to  the  CID  AOB  desk.  These  are 
hardcopy  messages,  forwarded  via  the  'tennis  shoe 
net’  -  i.e.,  hand  carried. 

The  AOB  Officer  in  Current  Plans  receives  the 
TACREPS,  sorts  them  by  mission  type,  plots  their 
routes  of  flight  on  his  wall  map,  and  updates  his 
tallies  of  enemy  missions  by  mission  type,  base  of 
origin,  and  aircraft  type.  He  uses  these  tallies  and 
plots  to  determine  significant  events  and  trends,  to 
identify  the  airbases  the  enemy  is  using,  and  to 
recommend  defensive  measures  and  targeting 
opportunities. 

Because  TACREPS  are  manually  produced, 
occasional  spelling  and  other  errors  came  through. 
TACREPS  may  be  as  much  as  15  or  20  minutes 
behind  real  time.  In  a  major  air  battle  there  may 
be  as  many  as  3000  TACREPS  per  day. 


OTHER  MESSAGES 

In  addition  to  TACREPS,  the  AOB  analyst 
makes  use  of  a  number  of  other  types  of  messages. 
Some  of  these  are: 

KILLSUM  (Kill  Summary):  This  is  a  list  of 
claims  of  destroyed  aircraft.  It  is  a  well 
formatted  message,  each  line  of  which  is  the  word 
CLAIMS,  followed  by  codes  for  the  type  of 
engagement,  number  of  aircraft  destroyed  on  the 
ground,  number  and  location  of  aircraft  engaged, 
and  so  on.  It  is  the  KILLSUM  which  provides  the 
AOB  officer  with  the  results  of  engagements  with 
enemy  aircraft. 

MISREP  (Mission  Report):  This  report  is 
based  on  a  quick  aircrew  debriefing.  It  provides 
early  assessment  of  the  mission’s  effectiveness, 
but  may  be  less  accurate  than  a  later,  more 
detailed  assessment. 

IPIR  (Initial  Photographic  Intelligence 
Report):  This  is  a  quick  interpretation  of 

reconnaissance  photography;  again,  it  may  be 
subject  to  correction  based  on  later  more  detailed 
photo  interpretation. 

/ 

RECCEXREP  (Reconnaissance  Exploitation 
Report):  This  is  the  detailed  result  pf  a 

reconnaissance  mission.  RECCEXREPS  are  fairly 
v^ell  formatted,  but  have  fields  which  are 
narrative  in  form,  as  well  as  the  all-purpose  AMPN 
(amplification)  field,  which  is  free  text.  For 
example:  "no  AOB  noted" 
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THE  ROLE  OF  DECISION  AIDS  FOR  AOB  ANALYSTS 

ESAU  will  provide  specific  capabilities  in 
support  of  the  AOB  analyst.  In  addition  to  those 
currently  planned,  there  are  opportunities  for  a 
number  of  advances  which  could  materially 
contribute  to  the  functioning  of  the  TACC.  In  this 
section  we  discuss  both  the  specific  capabilities 
currently  under  development  and  the  potential  for 
the  future. 

ESAU  Functionality 

Figure  2  shows  the  functional  flow  within 
ESAU,  illustrating  the  main  functional  capabilities 
of  the  system.  ESAU  will  operate  on  TACREPS, 
KILLSUMS  and  RECCEXREPS.  Each  incoming  message 
will  be  subjected  to  error  detection  and  correction: 
where  ESAU  Is  unable  to  confidently  deal  with  an 
erroneous  message,  user  interaction  will  be 
solicited. 

ESAU  will  associate  each  TACREP  and  each 
killed  aircraft  in  a  KILLSUM  with  the  corresponding 
track  in  its  internal  blackboard  database.  The 
principal  means  of  association  for  TACREPS  is 
their  internal  track  number,  which  is  known  to  be 


reliable.  The  association  of  killed  aircraft  with 
existing  tracks  will  be  accomplished  by  using 
expert  rules  for  selecting  the  most  appropriate 
candidate  associations.  These  rules  will  compare 
aircraft  types,  geometric  calculations  based  on 
observed  course  and  speed  in  the  TACREPS  vs 
location  and  time  of  the  engagement  reported  in  the 
KILLSUM,  and  so  on. 

Expert  rules  will  also  be  used  in  associating 
air  tracks  with  base  of  origin.  In  some  cases  the 
TACREPS  may  contain  this  information;  in  other 
circumstances  they  may  not,  and  the  association  of 
tracks  with  base  of  origin  has  to  be  made  in  the 
AOB  cell.  These  rules  will  use  similar 
considerations  of  reasonableness,  involving  the 
aircraft  and  support  capabilities  known  to  be  at 
candidate  bases  of  origin,  course  geometry 
compared  to  the  location  fo  the  candidate  base,  etc. 

Determination  of  the  base  of  origin  will 
enable  ESAU  to  recommend  changes  to  the  AOB 
based  on  enemy  losses  in  combat.  Similarly,  when 
aircraft  land  at  other  than  the  base  of  origin,  this 
is  evidence  of  a  redeployment,  since  the  Soviets 
seldom  use  recovery  bases  other  than  the  base  of 
origin  except  in  emergency.  ESAU  will  call  the 
user's  attention  to  the  possibility  of  redeployments 
appropriately. 
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Expert  rules  for  ESAU  are  not  yet  available, 
because  ESAU  is  currently  in  the  early  design  stage. 
However,  the  predecessor  program,  DAGR,  is  the 
baseline  for  ESAU.  To  provide  the  flavor  of  ESAU 
rules,  we  have  included  in  Figure  3  some  rules 
taken  from  DAGR.  ESAU  rules  will  be  more 
comprehensive  and  less  hardcoded,  but  their 
general  character  can  be  seen  in  the  examples  from 
DAGR. 

RECCEXREPS  will  report,  among  other  things, 
numbers  of  aircraft  observed  damaged  or  destroyed 
on  the  ground.  Of  course,  such  losses  need  to  be 
reflected  In  changes  to  the  AOB. 

A  major  part  of  the  AOB  analyst's  time  is 
spent  in  manually  sorting,  collating,  and  tabulating 
AOB  information.  Much  of  the  analyst's  time  is 
wasted  in  doing  tedious,  error  prone  clerical 
activity,  rather  than  doing  v,/hat  he  is  trained  to  - 
analysis.  For  instance,  at  a  recent  exercise,  over 
an  hour  was  spent  manually  producing  a  list  of 
enemy  air  assets  sorted  by  six  general  role  codes. 
The  AOB  included  specific  role  codes  for  each 
aircraft,  but  the  request  was  for  them  to  be 
consolidated  into  six  categories.  This  kind  of  'bean 
counting'  task  constitutes  a  significant  fraction  of 
the  analyst's  workload.  ESAU  will  provide  database 
management  support  to  allovy  much  of  this  bean 
counting  to  be  done  automatically. 


Example  AOB  Update  Rules 


EVENT 

SITUATION 

ACTION 

Kill  reported. 

Base  of  origin  (BOO) 
confidence  greater  than  zero. 

Subtract  one  aircraft  of 
most  likely  type 
and  make,  etc.  from 
the  most  likely  BOO. 

Landing  reported. 

Recovery  base  (RB) 
confidence  greater  than  zero. 

Subtract  one  aircraft  of 
most  likely  type 
and  make,  etc.  from 
the  most  lilccly  BOO, 
and  add  one  to  the 
most  likely  recovery 
base. 

Track! BOO  (or  RB)  match. 


REASON 

CONFIDENCE 

Distance  from  track  to  base  >  75  km 

No  match 

Distance  from  track  to  base  <  75  km 

(20-di£tancc)/!0 

Aircraft  type  (possible)  at  base 

+2 

Aircraft  type  not  (possible)  at  base 

-2 

Aircraft  type  unknown 

0 

Aircraft  model  (possible)  at  base 

+2 

Aircraft  model  not  (possible)  at  base 

-2 

Aircraft  model  unknovm 

0 

Aircraft  country  (possible)  at  base 

+2 

Aircraft  country'!  not  (possible)  at  base 

-2 

Aircraft  cotmtry  unknown 

0 

(Track  bearing  -  base  bearing)  <  30  degrees 

+2 

Future  Possibilities  for  Automated  AOB  Decision  Aid 

The  functional  capabilities  being  built  in 
ESAU  are  just  the  beginning  of  what  automated 
decision  aids  could  do  for  AOB  analysts.  Some  of 
the  major  benefits  would  accrue  merely  because 
the  process  is  computerized:  once  the  messages 
are  decoded,  databases  are  established  on  line,  and 
procedures  and  forms  are  implemented,  standard 
data  processing  technology  will  provide  substantial 
improvement  in  operational  effectiveness  of  the 
AOB  officer.  Beyond  that,  the  data  fusion 
technology  will  enable  the  system  to  assist  in 
tasks  that  require  some  degree  of  judgment,  such 
as  the  recognition  of  tactically  significant  events 
and  patterns  of  behavior. 

Various  bean  counting  summaries  are 
currently  produced  manually,  including  tables  of 
sortie  rates  by  airbase  and  by  mission  type. 
TACREPS,  which  are  printed  on  slips  of  paper,  are 
physically  sorted  by  mission  type  and  aircraft  type. 
Clearly  these  processes  can  be  done  electronically. 
Moreover,  elementary  graphics  capabilities  will  be 
able  to  provide  charts  showing  such  things  as 
sortie  generation  over  time,  which  is  not  shown 
graphically  at  present. 

Just  the  maintenance  of  the  TACREP  log 
would  be  a  big  improvement.  AOB  officers  track 
enemy  activities  for  PCA  prediction:  there  is  no 


KilllBOO  Match 


REASON 

CONFIDENCE 

Distance  from  track  to  base  >  75  km 

No  match 

Distance  from  track  to  base  <  75  km 

(20-distance)/10 

Aircraft  type  at  base 

+2 

Aircraft  type  not  at  base 

-2 

Aircraft  type  unknown 

0 

Aircraft  model  at  base 

+2 

Aircraft  model  not  at  base 

-2 

Aircraft  model  unknown 

0 

Aircraft  country  at  base 

+2 

Aircraft  countiyl  not  at  base 

-2 

Aircraft  country  unknown 

0 

Means  of  destrucfion=Fighter 

+2 

Atiacker/targct  angle  <  30  degrees 

+1 

KiWTrack  match 


REASON 

CONHOENCE 

Distance  from  track  to  base  >  75  km 

No  match 

Distance  ftom  track  to  base  <  75  km 

(20-distance)/10 

Aircraft  type  at  base 

+2 

Aircraft  type  not  at  base 

-2 

Aircraft  type  unknown 

0 

Aircraft  model  at  base 

+2 

Aircraft  model  not  at  base 

-2 

Aircraft  model  imknown 

0 

Aircraft  coimtry  at  base 

+2 

Aircraft  countiyl  not  at  base 

-2 

Aircraft  country  unknown 

0 

Track  reports  count  drop 

+2 

Figure  3.  Example  Rules  from  DAGR. 
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formal  logging  process.  AOB  officers  keep  written 
notes,  and  use  them  in  briefings.  The  availability 
of  machine  produced  logs  and  graphics  would  not 
only  relieve  the  analyst  of  much  tedious  manual 
work,  but  would  eliminate  a  potentially  serious 
source  of  errors,  since  the  final  product  is  now  the 
result  of  a  number  of  manual  transcriptions. 

Using  a  TACREP  history,  it  will  be  possible  to 
produce  graphic  outputs  of  track  corridors,  thereby 
assisting  in  the  recognition  of  patterns  of  enemy 
operations. 

The  AOB  analyst  must  consider  some  normal 
expectable  sequences  of  events  rather  than 
consider  the  arrival  of  each  individual  TACREP 
independently.  For  example,  an  offensive  counter¬ 
air  attack  will  be  quickly  followed  by  an  INFLTREP 
(inflight  report),  which  is  based  on  the  aircrew's 
verba!  report  of  results  while  still  airborne.  When 
the  aircrew  is  debriefed,  on  landing,  the  resulting 
message  is  the  MISREP,  which  is  less  timely  but 
more  comprehensive  and  accurate  than  the 
INFLTREP.  Later,  if  reconnaissance  is  ordered, 
there  will  be  a  similar  sequence  consisting  of  the 
IPIR  and  RECCEXREP.  The  result  of  all  this  Is  the 
battle  damage  assessment  (BDA).  Now  all  of  this 
takes  time,  and  during  the  interval  the  AOB  analyst 
knows  that  the  AOB  is  out  of  date,  and  may  know 
something  about  the  change  to  be  expected.  One 
hears  remarks  like  "we  haven't  gotten  our  BDA  yet, 
so  we  know  there  are  fewer  airplanes  but  we  don't 
know  how  many  fewer."  Thus  there  is  a  role  for  the 
interpretation  of  the  current  estimates,  based  on 
knowledge  of  predictable  sequences  of  events. 


An  automated  aid  should  help  prepare 
messages.  For  example,  "  Enemy  Losses  and 
Replacements"  is  currently  produced  manually;  It 
is  simply  a  tabulation  showing  type  of  aircraft, 
initial  number,  number  destroyed,  number  of 
replacements,  resulting  number  available.  Other 
messages  contain  free  text  summaries,  followed 
by  tabulations;  incorporation  of  automatic 
tabulations  via  an  integrated  message  preparation 
aid  would  materially  help  the  users. 


CONCLUSIONS 

Fusion  is  an  integral  part  of  the  AOB  function 
in  the  TACC.  The  CID  Intelligence  Officer's  role  is 
just  fusion:  the  combination  of  diverse  forms  of 
incoming  messages  and  other  information  to  form 
an  up  to  date,  detailed,  well  supported  estimate  of 
enemy  dispositions,  capabilities  and  intentions. 

Fusion  is  accomplished  mainly  by  manual 
methods.  At  present  there  are  essentially  no 
electronic  aids  available  to  the  AOB  officer,  other 
than  message  handling  systems.  Moreover,  since 
the  wall  map  and  grease  pen  are  central  to  the 
officers'  ability  to  discuss  the  situation 
effectively  with  each  other,  it  is  likely  that  at 


least  the  wall  map  will  remain  a  manual  function 
for  some  time. 

Fusion  is  substantially  supported  by  easily 
automated  bean  counting  operations.  Database 
functions  of  sorting,  counting,  updating, 
correlating,  and  charting  can  easily  replace 
significant  amounts  of  manual  labor,  leaving  more 
think  time  for  the  officers,  reducing  the  required 
staff  size,  and  eliminating  manual  errors  of 
omission  and  transcription. 

Expert  system  technology  can  support  the 
judgmental  aspects  of  fusion  in  the  TACC. 
Introduction  of  new  weapons  into  the  theater, 

redeployments,  navigational  corridors,  changes  In 
patterns  of  behavior,  all  can  be  called  to  the 
attention  of  the  user.  Using  expert  system 
techniques,  the  system  can  suggest  possibilities 
and  weigh  alternatives  in  much  the  same  way  that 
the  user  himself  does,  and  thereby  function  as  a 
supportive  colleague. 
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During  1988,  the  first  phase  of  the  OSIS  (Ocean  Surveillance  Information  System)  Baseline  Upgrade  (OBU) 
system  is  being  deployed  to  the  five  OSIS  data  fusion  and  correlation  sites.  As  a  major  upgrade  of  a  currently 
operational  capability,  a  conservative  set  of  laboratory  and  field-proven  data  fusion  correlation  and  tracking 
algorithms  has  been  Implemented  In  this  system. 

Data  fusion  systems  can  be  categorized  based  on  four  attributes: 

1 .  Variety  of  sensor  Input  -"similar  sources"  (e.g.  receipt  of  data  for  correlation  from  many  acoustic  sensors) 
or  “dissimilar  sources" 

2.  Closeness  to  the  sensor(s)  -  “embedded"  (existing  as  a  part  of,  or  in  close  proximity  to,  sensors;  e.g. 
aircraft  cockpit  correlation  of  radar  and  EO  sensors)  or  “autonomous"  (away  from  the  sensor) 

3.  Destination  of  fused  results  -"localized  receptor"  (results  do  not  go  beyond  immediate  system/operators)  or 
“receptor/transmitter"  (results  are  transmitted  beyond  the  Immediate  fusion  system). 

4.  Sensor  sample  rate  -  “oversampled"  (updates  at  twice  target  maneuver  rate)  or  “undersampled"  (updates 
less  than  maneuver  rate) 


The  OBU  system  as  a  “dissimilar  source",  “autonomous",  “receptor/transmitter",  “undersampled",  correlation 
system  must  operate  In  the  most  complex  data  fusion  environment.  This  paper  will  explore  some  of  the  Issues 
encountered  Integrating  disparate  algorithms  operating  on  dissimilar  (both  In  data  content  and  security  classification) 
sensor  data  under  the  roof  of  a  single  system.  In  addition  to  algorithm  Implementation,  complexities  In  the  areas  of 
message  handling,  analyst  Interaction,  data  management,  system  recovery,  testing,  handling  of  multi-level  security, 
and  throughput  performance  had  to  be  overcome  In  order  to  turn  data  fusion  theory  and  proven  algorithms  Into  an 
Integrated  operational  system. 

The  paper  will  also  discuss  the  flexibility  required  In  the  operation  of  the  fusion  algorithms  as  a  result  of  both  the 
dissimilar  nature  of  the  sensors  and  the  differing  nature  of  the  fusion  problem  at  geographically  separate  site 
locations.  The  OBU  system  architecture  not  only  provides  growth  In  the  hardware  and  software  dimensions,  but  also 
supports  flexibility  and  growth  In  the  algorithm  dimension.  The  currently  Implemented  system  provides  both  serial 
(execution  of  a  sequence  of  algorithms  on  a  single  Incoming  contact  report)  and  parallel  (execution  of  different 
sequences  of  algorithms  for  different  Incoming  contact  reports)  adjustment  of  the  fusion  algorithms  using  a  process 
called  “channelization.” 
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OVERVIEW 

This  paper  describes  theoretical  and  practical  as¬ 
pects  of  conceiving,  building  and  deploying  a  large  data 
fusion  system.  The  discussion  Is  presented  In  the  context 
of  development  of  the  Navy’s  Ocean  Surveillance  Infor¬ 
mation  System  (OSIS)  Baseline  Upgrade  (OBU). 


DATA  FUSION  SYSTEM  EVOLUTION 

The  generic  evolution  of  a  data  fusion  system  devel¬ 
opment  Is  shown  in  Figure  1 .  The  flow  within  the  figure 
and  the  Influence  of  practical  considerations  on  the  evo¬ 
lution  from  algorithms  and  architecture  to  an  operational 
system  are  the  basis  for  this  paper.  This  evolution  starts 
with  consideration  of  the  fusion  environment  (called  “Na¬ 
ture  of  Correlation  Problem”  In  the  figure)  and  with  “Mis¬ 
sion  and  Operational  Constraints",  Three  components  of 
the  data  fusion  approach  are  developed  by  applying 
“Technology"  to  the  challenge  of  the  environment  and 
the  operational  mission:  algorithms,  data  fusion  architec¬ 
ture,  and  system  design.  The  “Nature  of  the  Correlation 
Problem",  “Mission  and  Operational  Constraints"  and 
“Technology"  subjects  in  Figure  1  represent  knowledge 
about  the  fusion  problem  and  potential  methods  of  solu¬ 
tion.  The  three  boxed  Items  derived  from  them  represent 
the  Initial  technical  definition  of  the  solution. 

Once  algorithms  are  selected,  architecture  is  deter¬ 
mined  and  system  design  Is  complete,  the  system  must 
be  built  and  tested.  A  number  of  practical  “Development 


Realities"  either  are  discovered,  or,  If  previously  known, 
begin  to  have  more  influence.  As  represented  In  Figure 
1 ,  these  realities  and  the  resulting  “Development  Conse¬ 
quences"  affect  the  realization  of  the  final  system  by 
transforming  or  modifying  portions  of  the  design  to 
achieve  a  workable  “Operational  System". 

The  major  section  headings  of  this  paper  will  be 
keyed  to  Figure  1.  With  the  exception  of  “System  De¬ 
sign"  and  “Operational  System",  which  are  topics  too 
large  for  a  paper  of  this  length,  each  subject  shown  in  the 
figure  will  be  discussed.  At  the  end  of  this  paper,  some 
planned  and  possible  future  directions  for  the  OBU  sys¬ 
tem  will  be  discussed. 


NATURE  OF  THE  CORRELATION  PROBLEM 

Four  environmental  factors  can  be  used  to  catego¬ 
rize  a  data  fusion  system:  source,  location,  destination 
and  Information.  Together  they  define  the  “nature  of  the 
correlation  problem."  Figure  2  addresses  three  of  these 
factors.  In  the  figure,  fusion  system  SI  receives  Informa¬ 
tion  from  a  single  type  of  source  (or  sensor)  while  sys¬ 
tem  S2  receives  Information  from  more  than  one  type  of 
source;  this  Is  the  source  factor.  SI  Is  a  “similar  source” 
system,  S2  a  “dissimilar  source"  system.  The  complex¬ 
ity  In  several  areas  of  the  fusion  system  can  be  directly 
related  to  the  source  factor:  preprocessing  before  data 
fusion,  data  fusion  algorithms,  and  analyst  tools.  In  addi¬ 
tion,  if  the  system  must  pass  fusion  results  to  externa! 
users,  dissimilar  security  classifications  between  sources 
can  add  to  the  difficulty  of  producing  reports. 
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Figure  1 .  Evolution  of  Data  Fusion  System  Development 
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Figure  2.  Relationships  of  Data  Fusion  Systems  with  Data  Sources  and  Consumers 


The  second  factor,  the  location  factor,  addresses 
the  proximity  of  the  fusion  system  to  the  source.  Proxim¬ 
ity  can  have  a  physical  dimension,  a  functional  dimension, 
or  both.  In  the  figure,  both  S1  and  S2  are  collocated  with 
the  sensors  and  represent  systems  “embedded”  with  the 
sensors.  These  systems  deal  with  sensor  Information  In  a 
raw  or  partially  processed  form,  and  receive  timely  and 
time-ordered  data  Input.  In  contrast,  S3  Is  an  “autono¬ 
mous”  fusion  system  located  away  from  the  sensors.  It 
receives  processed  and  translated  (typically  Into  stan¬ 
dard  communication  messages)  data  derived  from  sen¬ 
sor  Information;  the  data  Is  usually  not  as  timely  as  that 
received  by  the  embedded  system.  The  amount  of 
preprocessing  required  before  data  fusion  takes  place 
and  the  form  of  the  data  fusion  algorithms  (Information 
may  arrive  out  of  order)  are  related  to  the  location  fac¬ 
tor. 

For  systems  S1 ,  S2  and  S3  the  consumer  of  the 
fusion  results  resides  at  the  fusion  system,  while  for  S4 
the  consumer  is  at  another  location.  This  third  factor, 
the  destination  factor,  can  be  used  to  categorize  a 
fusion  system  as  either  a  “localized  receptor”  or  a  “re¬ 
ceptor/transmitter”.  The  destination  factor  determines 
the  need  for  capabilities  In  the  fusion  system  to  post¬ 
process  fusion  results  Into  reports  that  can  be  sent  to 
consumers.  It  can  also  impact  system  data  management 
by  requiring  special  security  handling  In  fusion  systems 
having  dissimilar  security  classification  source  Input. 

The  final  factor,  the  information  factor,  deals  with 
the  quality  of  input  information.  Both  accuracy  of  infor¬ 
mation  and  sample  rate  with  respect  to  target  maneuver 
rates  must  be  considered  when  selecting  data  fusion  al¬ 
gorithms.  Adequately  or  oversampled  systems  can  use 
classical  correlation  and  tracking  algorithms.  “Under¬ 
sampled”  systems  must  rely  on  more  involved  means  In 
an  attempt  to  fill  gaps  In  the  Information.  This  results  in 
the  need  for  sophisticated  fusion  algorithms  and  analyst 


tools  for  resolving  Inconsistencies.  The  Information  factor 
Is  an  Important  driver  In  developing  the  algorithms;  the 
source,  location  and  destination  factors  can  be  as  Impor¬ 
tant  when  developing  a  system. 

The  OSIS  (Ocean  Surveillance  Information 
System)  Baseline  Upgrade  (OBU) 

During  1988,  the  first  phase  of  the  OBU  system  Is 
being  deployed  to  the  five  OSIS  data  fusion  and  correla¬ 
tion  sites.  As  a  “dissimilar  source”,  “autonomous”,  “re¬ 
ceptor/transmitter”,  “undersampled”  correlation 
system.  It  must  operate  In  one  of  the  most  complex  data 
fusion  environments.  An  additional  factor  influencing  the 
OBU  development  was  an  existing  operational  capability: 
the  OSIS  Baseline  System  (OBS).  It  necessitated  an  ap¬ 
proach  that  did  not  cause  a  major  perturbation  In  the 
ability  of  the  sites  to  continue  operations.  To  avoid  dis¬ 
ruption  of  site  operations  from  algorithm  Induced  anoma¬ 
lies  or  failures,  a  conservative  set  of  laboratory  and 
field-proven  algorithms  based  on  existing  technology  was 
selected  for  the  OBU  system.  The  challenge  of  the  data 
fusion  environment  was  addressed  in  the  OBU  architec¬ 
ture;  several  algorithms  were  combined  in  a  flexible  and 
synergistic  fashion. 

In  the  case  of  the  OBU  system,  the  “dissimilar 
source” ,  “autonomous” ,  “receptor/transmitter”  envi¬ 
ronmental  factors  are  established  from  the  OSIS  Mission 
and  Operational  Constraints  (discussed  In  the  next  sec¬ 
tion).  The  final  factor,  the  quality  of  the  sensor  data,  is 
common  to  most  autonomous  ocean  surveillance  sys¬ 
tems.  The  Information  Is  generally  greatly  under¬ 
sampled,  of  varying  degrees  of  accuracy,  often  received 
in  a  different  order  than  acquired,  and  representing 
knowledge  about  disparate  attributes  of  the  target  plat¬ 
form.  This  disparity  can  be  Illustrated  by  considering  the 
Information  that  might  be  received  for  a  submarine.  One 
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Figure  3.  OBU  Services  a  Variety  of  Sources  and  Consumers 


report  (called  a  contact)  might  contain  a  location  and 
platform  Identification  (visual  sighting).  It  might  be  fol¬ 
lowed  by  a  contact  containing  acoustically  derived  pa¬ 
rameters  and  a  line  of  bearing  (ACINT),  followed  by  a 
location  estimate  and  measurement  of  radar  parameters 
(ELINT) ,  followed  by  a  location  estimate  and  a  radio  call 
sign  (SIGINT),  This  information  must  be  combined  to 
provide  an  overall  picture  of  the  characteristics,  location 
time  line  and  Identity  of  the  specific  platform. 


OSIS  MISSIONS  AND  OPERATIONAL 
CONSTRAINTS 

The  Ocean  Surveillance  Information  System  (OSIS) 
Is  an  essentia!  element  of  the  Navy  Command  and  Con¬ 
trol  System  (NCCS)  ashore.  The  Navy  Command  and 
Control  Plan  states  the  OSIS  mission  as  “receive,  proc¬ 
ess,  and  disseminate  timely  all  source  ocean  surveillance 
Information  on  mobile  targets  of  Interest,  above,  on,  and 
under  the  oceans,  to  the  Navy  and  other  services  at  all 
levels  of  command.”  There  has  also  been  Increasing  In¬ 
terest  In  the  support  of  Navy  power  projection  missions 
through  monitoring  coastal  zone  land-based  targets. 
There  is  a  large  number  of  Information  sources  for  the 
targets  of  Interest  and,  within  the  mission  definition,  a 
large  number  of  potential  consumers  of  the  results  of  fu¬ 
sion.  As  a  result,  the  OBU  system  must  receive  and 
process  positional  and  electronic  intelligence  reports  for 
surface  ships,  submarines,  aircraft,  and  points  ashore.  It 
must  support  the  correlation,  assessment,  and  evalu¬ 
ation  of  the  reported  Information,  and  generation  of  the 
necessary  Integrated  Ocean  Surveillance  Products 
(OSP)  to  support  both  afloat  and  shore  In-theater  levels 
of  command.  The  OSP  Includes  both  scheduled  and 
event-by-event  (EBE)  reports.  Figure  3  Illustrates  the 
variety  of  sources  and  consumers  the  OBU  system  must 
service . 


The  OSIS  is  a  distributed  system  which  is  geographi¬ 
cally  oriented  to  support  fleet  commanders  and  afloat 
forces.  As  a  result,  the  OBU  system  Is  being  deployed  to 
five  operational  sites,  each  having  somewhat  different 
availability  of  sensor  data  and  emphasis  within  the  overall 
OSIS  mission.  As  an  example,  one  site  might  place  first 
emphasis  on  air  platforms  while  another  might  place 
greater  emphasis  on  afloat  platforms.  This  emphasis  Is 
subject  to  change  depending  on  the  tactical  situation.  In 
supporting  multiple  sites,  each  performing  multiple  func¬ 
tions,  the  OBU  system  must  provide  the  flexibility  to  sup¬ 
port  multiple  analysts  doing  similar  but  different  tasks. 

Several  key  program  level  objectives  derived  from 
the  OSIS  mission  guided  the  OBU  development  and  were 
significant  factors  In  defining  the  data  fusion  approach: 
provide  a  multi-source  Integrated  locational  data  base, 
provide  automated  event-by-event  reporting  with  no  hu¬ 
man  in  the  loop,  and  allow  the  Integration  of  data  of  mul¬ 
tiple  security  levels  and  classifications.  Including  Sensitive 
Compartmented  Information  (SCI).  An  additional  pro¬ 
grammatic  objective  was  to  establish  a  system  backbone 
to  support  future  growth.  That  growth  may  occur  In  sev¬ 
eral  dimensions,  from  Increased  quantities  of  input  data 
and  Increased  variety  of  Input  sources,  to  more  specific 
handling  of  the  characteristics  of  Important  subsets  of 
the  target  population.  By  developing  a  robust  and  flexible 
OBU  system,  relatively  modest  additional  Investment  can 
evolve  the  system  to  meet  changing  environments  and 
emphasis. 

Many  of  the  mission  and  program  objectives  can  be 
summarized  In  a  single  primary  concept:  a  single  data 
base  consisting  of  fused  Information  on  all  platforms, 
from  all  sources,  at  all  security  levels  that  Is  available 
both  for  display  and  analysis  by  OSIS  analysts,  and  for 
transmission  in  a  tailored  form  to  consumers  of  the  OSP. 
It  is  this  concept  which  guided  the  definition  of  the  OBU 
system.  Within  a  given  operational  site,  there  also  exist 
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several  small  systems  currently  accomplishing  portions  of 
the  overall  ocean  surveillance  job.  These  systems  exist 
as  recently  developed  prototypes  which  have  unique  ca¬ 
pabilities  or  applications.  Combination  of  these  capabili¬ 
ties  with  the  OBU  system  to  produce  an  Integrated 
picture  for  the  analyst  is  a  part  of  this  primary  concept. 


TECHNOLOGY 

Referring  to  Figure  1 ,  once  the  Nature  of  the  Corre¬ 
lation  Problem  and  the  Mission  and  Operational  Con¬ 
straints  are  defined,  technology  is  applied  to  develop 
algorithms,  architecture  and  a  system  design.  The  tech¬ 
nology  base  for  the  fusion  system  Includes  hardware,  al¬ 
gorithms,  and  software  engineering  disciplines.  In  the 
case  of  OBU,  no  hardware  or  algorithm  technology  de¬ 
velopment  was  undertaken.  Emphasis  was  placed  on  the 
exploitation  of  existing  technologies  within  the  framework 
of  a  unified  system.  New  software  engineering  ap¬ 
proaches  were  required  for  the  development  of  a  multi¬ 
level  security  accreditable  large-scale  software  system. 
Existing  technology  for  control  and  coordination  of  appli¬ 
cation  processing  on  a  set  of  local  distributed  computing 
nodes  was  applied  to  solve  the  combined  need  for  very 
high  system  availability  and  support  of  multiple  analysts  In 
a  multi-task  environment. 


SELECTED  ALGORITHMS 

The  goal  of  algorithm  selection  In  a  data  fusion  sys¬ 
tem  Is  to  use  the  best  algorithms.  The  factors  that  de¬ 
termine  “best”  vary  from  system  to  system.  Given  the 
disparity  of  Input  sources  and  platform  types  the  OBU 
system  must  handle,  there  is  no  single  algorithm  that  can 
correlate  all  of  the  Information;  algorithm  selection  was 
the  determination  of  a  synergistic  set  of  algorithms.  In¬ 
formation  received  by  the  OBU  system  Is  In  the  form  of 
contact  reports  containing  one  or  more  of  the  following 
categories  of  Information: 

1 .  Unique  Attribute  -  data  which  specifically  Identifies 
either  a  platform  (like  ship  name)  or  a  track  (this  re¬ 
port  Is  one  of  a  series  belonging  to  the  same  plat¬ 
form) 

2.  Physical  Data  -  items  like  ship  class  or  country  which 
provide  partial  Identification  of  a  platform 

3.  Parametric  Data  -  measured  parameters  relating  to 
a  platform;  in  the  OBU  system  these  refer  to  meas¬ 
ured  radar  parameters  (FLINT) 

4.  Location  Data  -  estimates  of  location  with  an  associ¬ 
ated  containment  ellipse.  These  data  Items  occur  in 
reports  of  the  previous  three  categories ,  but  can  also 
be  received  alone. 

These  four  categories  of  Information  are  used  within 
the  OBU  system  In  a  hierarchical  fashion,  processed  In 
the  order  listed.  The  only  exception  to  the  ordering  Is 
early  screening  of  platform  tracks  by  location;  this  avoids 
attempted  correlation  of  a  contact  report  to  platforms 
not  capable  of  being  at  the  contact  location  at  the  re¬ 
ported  time.  The  result  of  ordered  processing  is  sequen¬ 
tial  application  of  separate  correlation  algorithms  for 
unique  attribute,  parametric,  and  spatial  Information, 
with  the  use  of  physical  and  location  information  to  elimi¬ 
nate  incompatible  correlation  candidates. 

Unique  Attribute  Data 

Rules  are  used  for  matching  this  Information  to  de¬ 
termine  contact  correlation  to  platform  tracks,  or  for 


elimination  of  platform  tracks  from  further  consideration. 
The  rules  were  developed  by  examining  other  systems 
and  studying  message  traffic;  they  allow  the  OBU  system 
to  take  advantage  of  processing  performed  at  the  sensor 
(which  can  be  as  simple  as  reading  a  hull  number  off  a 
ship  through  a  pair  of  binoculars  or  as  complex  as  track¬ 
ing  a  target  through  processing  radar  returns) .  Although 
the  Individual  rules  are  straightforward  and  do  not.  In  the 
normal  use  of  the  terminology,  constitute  “algorithms”, 
layering  of  this  correlation  process  on  top  of  the  other 
correlation  algorithms  Is  an  Important  aspect  of  the  over¬ 
all  approach.  This  apparently  simplistic  correlation 
method  Is  closely  coupled  to  the  reporting  rules  and  con¬ 
ventions  enforced  In  the  reporting  system.  Changes  In 
message  formats,  management  of  platform  Identifiers, 
or  the  existence  of  exceptions  to  defined  meanings  of 
message  fields  all  serve  to  perturb  the  correlation  rules 
which  use  unique  attribute  information. 

The  initial  list  of  unique  attributes  for  correlation  or 
track  candidate  elimination  used  In  the  OBU  system  Is 
given  In  Table  1 .  As  a  consequence  of  reporting  conven¬ 
tions  and  sensor  management  of  these  attributes,  not  all 
can  be  treated  uniformly  In  the  correlation  process.  This 
distinction  is  Indicated  in  the  table  under  the  columns  la¬ 
beled  Number  of  Values  and  Qualifiers.  Unique  attributes 
having  one  value  and  no  qualifiers  are  sufficient  to  cause 
correlation  of  a  contact  to  a  track  in  the  case  of  a 
match,  or  elimination  of  the  track  from  further  considera¬ 
tion  (for  parametric  and  spatial  correlation)  In  case  of  a 
mismatch.  Attributes  having  more  than  one  value  and  no 
qualifiers  can  cause  correlation  on  a  match,  but  do  not 
disqualify  the  track  from  further  consideration  when  there 
Is  a  mismatch.  Attributes  having  “plus”  qualifiers  are 
treated  as  concatenations  of  the  attribute  and  the  qualifi¬ 
er;  this  concatenation  produces  a  combined  value  for 
both  correlation  and  track  elimination  purposes.  For  ex¬ 
ample,  a  contact  report  with  Pendant  Number  805  and 
flag  US  will  correlate  with  any  track  having  the  same  Pen¬ 
dant  Number/flag  combination.  All  tracks  having  either  a 
different  Pendant  Number  or  flag  are  disqualified  from 
further  consideration.  Attributes  having  “with”  qualifiers 
are  treated  as  concatenations  for  correlation  purposes, 
but  track  disqualification  occurs  only  when  two  different 
values  of  the  attribute  occur  for  a  given  value  of  the 
qualifier.  For  example,  a  contact  report  with  Raid  Num¬ 
ber  123  and  command  XYZ  will  correlate  only  with  a  track 
having  the  same  Raid  Number/command  combination. 
Tracks  with  differing  Raid  Numbers  will  be  disqualified  for 
further  consideration  only  If  they  have  a  command  value 
of  XYZ. 


Table  1 .  Initial  OBU  Unique  Attribute  Parameters 


Attribute 

Number 

Of 

Values 

Qualifiers 

NOSIC  ID 

1 

None 

(Radio)  Call  Sign 

5 

None 

Ship  Name 

1 

None 

Pendant  Number 

1 

Plus  Flag,  With  Type 

Precor  Track  Number  3 

With  Command 

Raid  Number 

3 

With  Command 

Trademark 

3 

With  Command 

PIF  Code 

3 

With  Command 

The  Initial  unique  attribute  definitions  for  correlation 
and  their  handling  will  be  modified  as  dictated  by  experl- 
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ence,  new  data  soi^rcas  or  changes  In  reporting  conven¬ 
tions. 

Physical  Data  and  Consistency  Tests 

Physical  data  Is  used  to  eliminate  Incompatible  cor¬ 
relation  candidates.  All  candidate  platform  tracks  having 
physical  attributes  (flag  or  country  code,  platform  cate¬ 
gory,  platform  class,  platform  type)  inconsistent  with  the 
contact  report  under  consideration  are  eliminated  from 
consideration  for  parametric  and  spatial  correlation.  All 
successful  unique  attribute  correlations  are  also  tested 
for  physical  data  consistency;  conflicts  are  reported  to 
an  analyst  for  resolution.  In  addition  to  the  physical  data, 
geofeasibility  (determined  from  location  and  maximum 
platform  category  speed)  and  allowable  platform  emitter 
complement  (determined  at  the  ship  class  level  from  a 
stored  table)  are  used  In  similar  consistency  and  elimina¬ 
tion  tests. 

Parametric  Data 


ELI  NT  measurements  of  radar  parameters  repre¬ 
sent  the  majority  of  the  parametric  Information  received 
by  OBU.  The  current  OBU  system  uses  only  this  type  of 
parametric  data;  acoustical  and  other  types  of 
parametric  data  are  not  processed  for  correlation  pur¬ 
poses.  Almost  all  ELINT  parametric  data  received  at  an 
OSIS  site  has  undergone  preprocessing  by  the  source  to 
determine  the  type  of  radar  reported;  the  radar  type  is 
Identified  by  the  SEDSCAF  or  ELINT  Notation  (ELNOT). 
Algorithms  Implemented  in  OBU,  therefore,  depend  on 
the  (correct)  existence  of  the  ELNOT  for  processing  the 
parametric  data. 

The  ELINT  processing  algorithm  selected  for  OBU 
was  developed  from  research  performed  at  NOSC  during 
the  early  1980's.  This  algorithm,  TERESA  (Target  Evalu¬ 
ation  and  Recognition  by  Extraction  of  Statistical  Attrib¬ 
utes),  is  a  “second  generation"  ELINT  correlation 
algorithm  that  uses  a  statistical  distribution  rather  than 
range-oriented  parametric  tracking  process. 

First  generation  ELINT  correlation  algorithms  are 
based  on  parametric  and  spatial  trend  tracking,  using 
analyst  managed  parametric  ranges  and  system  derived 
statistical  means  to  screen  and  score  candidates.  These 
algorithms  evolved  through  a  series  of  prototype  efforts 
(NEAT  -  Naval  ELINT  Analysis  Tool,  BELT  -  Basic  ELINT 
Tracker,  POST  -  Prototype  Ocean  Surveillance  Termi¬ 
nal)  which  used  experiences  vylth  the  Initial  algorithms  and 
overlaid  field  developed  heuristic  rules  on  the  analytical 
aspects  of  the  correlation  process. 

TERESA  uses  a  completely  statistical  approach  to 
ELINT  correlation.  This  algorithm  assumes  that  the  pa¬ 
rameters  of  an  emitter  observed  through  Independent 
measurements  follow  a  probability  distribution;  correla¬ 
tion  decisions  are  mathematically  derived  consistent  with 
the  assumed  statistical  model,  TERESA  Initially  uses  de¬ 
fault  parametric  distributions  derived  from  analysis  of  ob¬ 
servations  of  specific  categories  and  versions  of  radars. 
The  algorithm  allows  the  parameters  that  characterize 
the  distributions  to  evolve  as  samples  are  correlated  to 
specific  emitters.  In  this  manner,  the  system  “learns" 
the  characteristics  of  specific  emitters,  eliminating  the 
detailed  management  of  ranges  for  scoring  and  correla¬ 
tion  required  by  first  generation  algorithms. 

Each  generation  of  ELINT  algorithms  has  Its  sup¬ 
porters  and  evidence  seems  to  Indicate  that  each  out 


performs  tho  oti'isr  In  ceitain  circumstanoes.  Fieferences 
1  and  2  contain  a  discussion  of  these  algorithms  and  offer 
some  comparison  data.  From  these  algorithms  has 
evolved  DUET  (Developmental  Unified  ELINT  Tracker) 
which  Incorporates  both  NEAT  and  TERESA  algorithms 
(Reference  3) .  Data  is  routed  to  one  or  tho  other  of  the 
algorithms  based  upon  Its  characteristics.  It  has  not 
been  shown  that  DUET  would  offer  Improved  perform¬ 
ance  over  TERESA  In  the  OBU  environment;  this  could  be 
because  superiority  of  the  NEAT  or  TERESA  algorithm 
may  depend  on  the  specific  correlation  situation  rather 
than  the  characteristics  of  Individual  Incoming  data 
Items,  More  experience  with,  and  evaluation  of, 
TERESA  and  DUET  In  operational  environments  Is  neces¬ 
sary  to  make  this  determination. 

Location  Data 

Processing  location  data  Is  one  of  the  most  mature 
forms  of  correlation  processing  because  It  Is  applicable 
to  a  wide  variety  of  applications.,  Spatial  consistency  be¬ 
tween  a  reported  location  and  the  estimated  or  projected 
position  of  a  candidate  platform  Is  basic  to  many  correla¬ 
tion  systems.  Determination  of  estimated  or  projected 
positions  through  the  us©  of  tracking  techniques  is  a  fea¬ 
ture  of  systems  that  use  location  data  to  correlate  mov¬ 
ing  targets.  Many  contact  reports  received  by  the  OBU 
system  do  not  contain  unique  attribute  or  unambiguous 
parametric  Information  and  therefore  must  be  correlated 
based  primarily  on  location.  Possible  track  projection 
techniques  vary  from  dead  reckoning  the  last  reported 
position  to  Kalman  filter  based  projection.  Because  re¬ 
ports  at  an  “autonomous"  correlation  system  do  not 
necessarily  arrive  In  observed  order,  the  selected 
method  must  accommodate  projection  to  times  before 
or  after  the  most  recent  update  to  the  candidate  plat¬ 
form  position. 

A  sophisticated  spatial  processing  algorithm  known 
as  the  Maneuvering  Target  Statistical  Tracker  (MTST) 
has  been  developed  for  tracking  Naval  ships  (Reference 
4)  and  proven  through  a  series  of  demonstrations. 
MTST,  as  Implemented  In  the  OBU  system,  uses  a  four 
state  (latitude,  longitude,  speed,  and  heading)  model  of 
the  platform’s  estimated  location  Including  uncertainties 
associated  with  the  estimate.  This  estimate  Is  derived 
from  a  combination  of  position  only  and  position/velocity 
contacts.  The  model  assumes  that  uncertainties  for  both 
reported  position  and  velocity  are  known  or  can  be  esti¬ 
mated.  The  algorithm  also  incorporates  a  motion  model 
for  tracking  platforms  that  can  maneuver  between  obser¬ 
vations,  The  Integrated  Ornstein  Uhlenbeck  (lOU)  motion 
mode!  Is  applicable  to  “undersampied"  correlation  sys¬ 
tems.  In  the  mode!,  platforms  are  assumed  to  have  a 
characteristic  speed  and  a  typical  time  between  maneu¬ 
vers;  projected  velocity  Is  adjusted  to  account  for  possi¬ 
ble  maneuvers. 

The  MTST  aigorithrn  contains  an  additional  feature 
that  supports  synergistic  combination  with  Unique  Attrib¬ 
ute  and  ELINT  correlation  algorithms:  post  correlation 
aberrant  point  detection.  In  the  OBU  system  approach  to 
correlation,  contact  reports  are  processed  against  exist¬ 
ing  platform  tracks  yielding  one  of  three  correlation  re¬ 
sults:  hard  correlation  to  an  existing  track,  failure  to 
correlate  to  any  existing  track,  or  ambiguous  correlation 
to  more  than  one  track  requiring  resolution  by  an  analyst. 
This  is  known  as  the  “single  hypothesis”  approach;  a  cor¬ 
relation  decision  In  the  OBU  system  can  only  be  changed 
by  manual  Intervention.  This  allows  the  possibility  that  a 
report  having  a  poor  spatial  fit  to  a  track  v/i!l  correlate 
because  of  unique  attribute  or  ELINT  data.  A  poor  spatial 
fit  could  modify  the  track  so  that  the  projections  for  sub- 
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s©Qu©nt  correlation  opportunities  result  In  failure  to  prop¬ 
erly  correlate.  A  spatial  miscorrelation  can  have  the 
same  effect.  The  MIST  algorithm  corrects  this  situation 
by  post  correlation  aberrant  point  detection.  This  “soft” 
correlation  feature  allows  the  location  component  of  a 
contact  report  to  be  Ignored  In  the  update  of  estimated 
position  while  Incorporating  the  other  Information  into  the 
track.  The  location  component  can  be  determined  aber¬ 
rant  and  removed  either  for  a  newly  correlated  contact 
or  for  a  previously  correlated  contact  that  is  not  consis¬ 
tent  with  subsequent  correlations. 


A  current  debate  in  the  correlation  community  con¬ 
cerns  the  effectiveness  of  single  hypothesis  versus  multi¬ 
ple  hypothesis  approaches  to  correlation.  Multiple 
hypothesis  approaches  can  defer  making  a  “hard”  corre¬ 
lation  decision  until  the  receipt  of  new  and  more  definitive 
data,  and  can  retroactively  undo  a  prior  “soft”  correla¬ 
tion  of  a  report  to  a  track  by  considering  new  data.  The 
potential  advantage  of  such  algorithms  is  a  reduction  In 
the  number  of  ambiguous  correlations  requiring  analyst 
resolution.  The  disadvantage  Is  the  imm.aturlty  of  the  ap¬ 
proach  in  several  practical  aspects.  The  Initial  perform¬ 
ance  barrier  to  this  approach  has  been  addressed  in 
recent  theoretical  and  practical  studies:  how  to  effi¬ 
ciently  “prune”  the  list  of  alternate  scenarios  retained  to 
keep  it  within  the  bounds  of  current  processing  capability 
(Reference  5) . 


Even  assuming  that  recent  work  has  solved  the  prun¬ 
ing  problem,  there  remain  technical  concerns  inhibiting 
use  of  multiple  hypothesis  algorithms  in  an  OBU-llke  envi¬ 
ronment.  These  concerns  relate  to  “receptor/transmit¬ 
ter”  correlation  systems  that  report  results  to  distant 
consumers.  In  this  environment,  computer  resources  are 
expended  solving  the  security  situation  that  results  from 
combined  fusion  of  multiple  level  security  information  and 
automatic  reporting  of  results  to  external  consumers. 
Even  optimally  pruned  multiple  hypothesis  approaches  re¬ 
quire  significantly  more  computer  resources  than  do  sin¬ 
gle  hypothesis  approaches.  In  addition  the  “pruning” 
aspects  of  multiple  hypothesis  systems  make  their  con¬ 
sumption  of  computer  resources,  and  therefore  their 
timeliness  performance,  statistical  rather  than  deter¬ 
ministic.  Quantifying  throughput  performance  before  the 
system  Is  deployed  to  the  field  becomes  an  extremely 
difficult  task. 


Additional  concerns  about  multiple  hypothesis  algo¬ 
rithms  deal  with  “changed  calls”  and  recovery  from  fail¬ 
ure.  “Receptor/transmitter”  correlation  systems  must 
report  correlation  decisions  externally,  often  In  real  time. 
In  the  OBU  system  Important  ambiguities  are  resolved 
using  the  analyst’s  best  judgement.  This  judgement  can 
include  Information  not  available  to  the  computer.  Reso¬ 
lution  of  less  Important  ambiguities  can  be  deferred  until 
more  Important  ones  are  resolved.  A  multiple  hypothesis 
system  theoretically  treats  all  ambiguities  equally,  await¬ 
ing  further  reports  before  making  “hard  call”  decisions. 
Its  reporting  strategy  must  decide  either  to  only  report 
“hard  call"  correlations  or  to  also  allow  reporting  of  most 
probable  ambiguous  correlations.  The  first  strategy  re¬ 
quires  close  analyst  review  of  ambiguous  correlations  to 
decide  when  to  force  external  reporting.  This  review  can 
negate  much  of  the  potential  advantage  of  these  algo¬ 
rithms  In  off-loading  work  from  the  analyst.  The  latter 
strategy  requires  philosophies  and  mechanisms  (on  the 
part  of  both  the  fusion  system  and  the  product  con¬ 
sumer)  for  an  Increased  quantity  of  changed  correlation 
messages  to  the  consumer. 


Recovery  from  failure  (power,  hardware  or  soft¬ 
ware)  is  always  difficult  In  systems  which  require  accu¬ 
rate  retention  of  data  and  must  meet  throughput 
performance  requirements.  The  problem  of  complete  re¬ 
covery  of  both  the  data  base  and  the  current  processing 
report  stream  appears  to  be  much  more  complex  with  a 
multiple  hypothesis  approach.  Database  recovery  after 
failure  was  a  major  challenge  In  the  single  hypothesis 
OBU  development.  The  Impact  of  the  journaling  of  up¬ 
dates  to  ensure  recovery  required  Iterative  design  efforts 
to  meet  throughput  requirements.  Journaling  Is  neces¬ 
sary  due  to  the  “undersampled”  nature  of  the  sensor 
data  reaching  the  OBU  system;  It  cannot  afford  to  lose 
an  Input  report  because  another  report  on  that  platform 
may  not  be  received  for  some  time.  “Brute  force”  ap¬ 
proaches  to  recovery,  like  restarting  the  processing  of 
reports  from  an  Input  log,  have  to  be  avoided  In  a  “re¬ 
ceptor/transmitter”  system  so  that  communication  cir¬ 
cuits  and  consumers  are  not  flooded  with  duplicate 
messages.  Further  discussion  of  multiple  hypothesis  ap¬ 
proach  Issues  can  be  found  in  Reference  6, 


OBU  DATA  FUSION  ARCHITECTURE 

As  shown  In  Figure  1 .  the  OBU  data  fusion  architec¬ 
ture  was  derived  from  Mission  and  Operational  Con¬ 
straints,  the  Nature  of  the  Correlation  Problem  and 
Technology.  This  architecture  Is  the  result  of  five  princi¬ 
ples  which  guided  Its  definition. 

Principles  of  the  Architecture 

The  first  principle  Is  that  the  architecture  must  allow 
the  flexibility  to  tailor  the  correlation  process  to  a  site- 
specific  environment.  The  five  CSIS  sites  may  have  dif¬ 
ferent  types  of  platform  targets,  mixes  of  sensor  data  (a 
sensor  which  Is  a  prolific  provider  of  information  to  one 
site  may  be  a  meager  provider  to  another),  consumer 
reporting  needs  (some  consumers  need  timely  event-by¬ 
event  reports  while  others  need  only  periodic  summa¬ 
ries),  and  priorities  within  the  broad  overall  mission 
guidance. 

The  second  principle  Is  that  the  architecture  must 
provide  a  backbone  for  future  upgrades;  It  must  be  ex¬ 
tensible.  Extensibility  In  quantity  of  data  processed  and 
number  of  analysts  supported  Is  provided  by  the  distrib¬ 
uted  hardware  and  network  control  software.  The  OBU 
data  fusion  architecture  Is  also  extensible  In  the  algorithm 
dimension.  The  current  algorithms  can  be  adjusted  to 
process  different  subsets  of  the  data  In  different  fashions 
without  software  change.  The  architecture  also  supports 
the  Insertion  of  new  algorithms  that  are  optimized  for 
processing  subsets  of  the  data  without  significantly  per¬ 
turbing  the  existing  algorithms. 

The  third  principle  Is  the  adoption  of  a  “divide  and 
conquer”  philosophy  to  achieve  throughput  performance. 
Envisioned  as  an  all-source,  all-target  data  fusion  sys¬ 
tem,  OBU  was  designed  with  the  knowledge  that  Its  track 
database  could  become  large.  The  combination  of  ship, 
air  and  land  targets  could  result  in  many  thousand  tar¬ 
gets  of  interest.  Additionally,  the  use  of  all-source  Infor¬ 
mation  In  support  of  correlation  raises  the  possibility  of 
significant  fragmentation  of  platform  tracks.  Data  from 
sensors  reporting  poorly  identified  or  ambiguous  Informa¬ 
tion  can  start  tracks  that  cannot  be  extended  or  confi¬ 
dently  correlated  to  nearby  tracks.  As  a  result,  the 
system  must  protect  itself  from  the  performance  Impli¬ 
cations  of  a  large  number  of  tracks.  Tv/o  forms  of  pro¬ 
tection  are  provided:  a  site  adjustable  method  called 
“channelization”  which  avoids  comparing  all  received 
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contacts  to  ail  tracks  In  the  database,  and  a  flexible 
scheme  for  establishing  Initiation  strategies  that  can  con¬ 
trol  the  number  of  new  tracks  started. 

The  fourth  principle  Is  that  the  system  must  protect 
the  correlation  process  from  “bad"  report  data.  Sensors 
and  operators  reporting  observations  make  mistakes;  un¬ 
less  detected  these  mistakes  can  corrupt  the  correlation 
data  base.  The  OBU  system  Implements  aggressive  con¬ 
sistency  checking,  notifying  the  analyst  to  resolve  Incon¬ 
sistencies  detected  during  correlation.  The  philosophy  is 
that  It  is  better  to  Involve  an  analyst  to  resolve  an  ambi¬ 
guity  caused  by  bad  data  than  to  allow  the  bad  data  to 
diminish  the  possibility  of  future  successful  correlation. 

The  fifth  principle  results  from  system  availability  re¬ 
quirements  and  the  mismatch  between  analyst  and  auto¬ 
matic  software  decision  rates.  A  distributed  hardware 
and  software  architecture  not  only  eliminates  single 
points  of  hardware  failure  but  also  decouples  the  ana¬ 
lyst's  decision  process  from  that  of  the  automatic  soft¬ 
ware.  The  analyst  Is  given  time  to  reach  correlation 
decisions  without  significantly  slowing  down  the  automatic 
software.  The  automatic  software  is  able  to  make  deci¬ 
sions  at  Its  natural  rate,  and  to  provide  the  analyst  Infor¬ 
mation  necessary  to  keep  him  current. 

Definition  of  the  Architecture 


The  OBU  correlation  architecture  Is  divided  Into  two 
parts:  automatic  and  Interactive.  The  automatic  portion 
performs  correlation  and  tracking  based  on  the  algo¬ 
rithms  previously  Identified.  Results  of  successful  corre¬ 
lation  and  ambiguous  situations  are  communicated  to 
analysts,  operating  at  Independent  work  stations,  ac¬ 


cording  to  both  predefined  and  ad  hoc  criteria. 
Predefined  updates  are  by  automatic  update  to  the  work 
station  of  tracks  previously  retrieved  by  the  analyst  and 
“work  lists"  containing  ambiguity  notifications  while  ad 
hoc  updates  are  by  analyst  query. 

At  the  work  station,  the  analyst  has  available  a  cor¬ 
relation  support  tool  kit.  The  analyst  can  invoke  any  of 
the  algorithms  available  to  the  automatic  processing,  on 
any  subset  of  tracks  or  contacts  copied  to  the  work  sta¬ 
tion.  The  tracks  and  contacts  on  the  work  station  are 
temporary  copies  of  the  permanent  tracks  maintained  by 
the  automatic  portion  of  the  system.  The  automatic 
processing  continues  the  correlation  process  on  new 
data  with  no  Interference  from  the  analysis  performed  at 
the  work  stations.  Automatic  updates  to  the  temporary 
track  copies  are  communicated  to  the  work  station  to 
keep  the  analyst  working  with  current  data.  The  results  of 
analyst  manipulation  of  the  temporary  tracks  become 
permanent  only  when  the  analyst  specifically  requests  an 
update  to  the  master  copy  of  the  track.  The  fifth  princi¬ 
ple  of  the  architecture  (decoupling  analyst  and  automatic 
work  rates)  is  addressed  by  this  separation  of  analyst 
and  automatic  processing. 

The  data  flow  through  the  automatic  processing  can 
be  viewed  as  having  two  dimensions:  through  algorithms 
and  through  “channels" .  The  driving  throughput  perform¬ 
ance  factor  in  any  correlation  system  Is  the  number  of 
contact-to-track  comparisons  which  must  be  made  to 
achieve  a  successful  correlation.  Both  dimensions  of  the 
OBU  data  flow  address  this  factor.  The  first  dimension  is 
the  flow  of  data  through  the  algorithms  within  the  tradi¬ 
tional  correlation  and  tracking  functions  shown  In  Figure 
4:  namely,  contact-to-track  correlation,  track  update, 
track  Initiation,  and  track-to-track  correlation. 


Figure  4.  OBU  Correlation  Functions 
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Serial  Application  of  Fusion  Algorithms 

Within  the  contact-to-track  and  track-to-track  cor¬ 
relation  functions,  data  is  treated  serially  using  Unique 
Attribute,  ELINT  and  spatial  algorithms.  These  algo¬ 
rithms  are  applied  In  order  of  Increasing  computational 
complexity  and  expected  probability  of  correlation  suc¬ 
cess.  Combined  with  physical  data  consistency  tests, 
the  algorithm  application  order  serves  to  “winnow”,  at 
each  step  In  the  process,  the  remaining  track  candidates 
to  compare  to  the  contact  report.  More  computationally 
expensive  algorithms  are  employed  on  fewer  contact/ 
track  pairs,  thereby  Increasing  the  efficiency  of  com¬ 
puter  use,  processing  throughput  and  timeliness.  The 
third  (“divide  and  conquer”  for  performance)  and  fourth 
(protect  against  “bad”  data)  principles  of  the  architec¬ 
ture  are  supported  by  the  algorithm  application  order  and 
the  consistency  checking  performed. 

The  first  processing  performed  for  a  contact  report 
against  a  set  of  candidate  tracks  (obtained  by  a  process 
called  “channelization"  described  in  the  next  section)  Is 
unique  attribute  correlation.  This  processing  searches  for 
matches  between  the  contact  attributes  listed  In  Table  1 
and  the  candidate  tracks.  A  match  of  one  or  several  at¬ 
tributes  to  one,  and  only  one,  track  causes  a  correlation. 
A  mismatch  of  any  attribute  (subject  to  the  considera¬ 
tions  discussed  in  the  Selected  Algorithms  section) 
causes  the  track  to  be  eliminated  from  further  considera¬ 
tion,  Including  ELINT  and  spatial  processing.  A  match  of 
one  attribute  coupled  with  a  mismatch  of  another  attrib¬ 
ute,  or  the  occurrence  of  matches  to  more  than  one 
track,  causes  the  generation  of  an  ambiguity.  The  con¬ 
tact  cannot  belong  to  more  than  one  track;  if  unique  indi¬ 
cators  suggest  It  should,  either  the  contact  report  Is 
defective  or  the  track  data  base  contains  a  correctable 
track  fragmentation.  The  analyst  must  determine  which 
is  the  case. 

Before  a  track  update  Is  performed  resulting  from 
unique  attribute  correlation,  physical  data  and  emitter 
complement  consistency  checking  are  performed.  The 
track  and  contact  must  be  geofeasible;  It  must  be  possi¬ 
ble  for  the  platform  to  traverse  the  necessary  distance  In 
the  allotted  time.  There  also  must  be  no  disagreement 
between  physical  data  fields  In  the  track  and  the  contact; 
agreement  is  not  required  because  this  information  Is 
often  not  reported. 

When  unique  attribute  correlation  Is  not  successful, 
physical  consistency  checking  Is  used  to  reduce  the  can¬ 
didate  track  set  prior  to  performing  the  computer  re¬ 
source  expensive  ELINT  and  spatial  algorithms.  For 
ELINT  contacts,  an  allowable  emitter  complement  table 
Is  consulted  to  eliminate  candidate  platform  tracks  be¬ 
longing  to  classes  that  cannot  carry  the  reported  emit¬ 
ter. 

ELINT  correlation  Is  performed  using  the  TERESA 
algorithm  on  contacts  containing  emitter  data  from  previ¬ 
ously  calibrated  sources  and  ELINT  Notations.  Those 
ELINT  contacts  not  qualifying  for  TERESA  processing  are 
treated  as  location  data  and  passed  to  the  spatial  algo¬ 
rithms.  The  TERESA  algorithm  returns  one  of  three  re¬ 
sults:  a  hard  call  assignment,  an  ambiguity,  or  a  new 
emitter  determination.  Both  hard  call  assignments  and 
new  emitter  determinations  are  based  purely  on  ELINT 
Information,  without  regard  to  location  data  (previous 
consistency  processing  determined  that  the  candidate 
tracks  are  geofeasible) .  When  an  ambiguity  Is  generated, 
the  system  will  use  the  spatial  correlation  algorithm  to  at¬ 
tempt  to  resolve  the  ambiguity.  Likewise,  when  a  new 


emitter  determination  Is  made,  spatial  algorithms  are 
used  to  attempt  to  find  a  platform  track  consistent  with 
the  emitter.  A  platform  track  can  carry  up  to  8  emitters. 

Spatial  correlation  algorithms  are  used  If  unique  at¬ 
tribute  and  ELINT  correlation  are  not  successful.  The 
MTST  algorithm  is  being  Implemented  In  the  OBU  system 
In  two  phases.  The  current  system  uses  the  Kalman  filter 
and  the  decaying  speed  lOU  motion  model  for  track  pro¬ 
jection.  The  backward  smoothing  (allowing  more  precise 
Interpolation  for  out-of-sequence  correlation)  and  aber¬ 
rancy  removal  portions  of  the  MTST  approach  are  cur¬ 
rently  In  development.  The  value  of  aberrancy  removal, 
given  the  hierarchical  nature  of  the  OBU  correlation  algo¬ 
rithms,  Is  discussed  In  the  Selected  Algorithms  section. 

Parallel  Adjustment  of  the  Fusion  Process 

The  second  dimension  (through  channels)  to  the 
data  flow  Is  Illustrated  In  Figure  5.  The  first  three  princi¬ 
ples  of  the  architecture  (flexibility,  extensibility,  and  “di¬ 
vide  and  conquer”  for  performance)  are  accommodated 
through  the  process  called  “channelization.”  Channeliza¬ 
tion  Is  the  partitioning  of  Incoming  contact  reports  and 
tracks  In  the  data  base  Into  subsets  for  processing.  Each 
channel  is  defined  by  a  query  statement  formed  from  a 
logical  combination  of  contact  and  track  attributes.  Con¬ 
tacts  are  placed  into  a  channel  by  comparison  to  the 
channel’s  query  statement.  The  same  query  statement 
Is  used  to  automatically  retrieve  tracks  for  potential  cor¬ 
relation  to  the  contact.  Following  Is  an  example  of  a 
query  statement: 

Channel  1:  [[CAT  EQ  NAV]  AND  NATO  AND  MED] 
NATO:  [”IC”  LE  CC  LE  “TU”] 

MED:  IN  RECTANGLE  [3000N,  4600N, 
00600W,  04000E1 

Query  statements  can  be  nested  as  shown  In  this  exam¬ 
ple,  where  the  terms  NATO  and  MED  are  defined  by 
separate  query  statements.  The  example  query  state¬ 
ment  will  route  contacts  to,  and  retrieve  tracks  for, 
channel  1  processing  that  have  a  physical  data  attribute 
of  category  NAV  (Naval),  a  reported  country  code  that 
falls  In  a  list  between  1C  (Iceland)  and  TU  (Turkey) ,  and  a 
position  within  a  lat/long  box  defined  by  the  limits  In  the 
RECTANGLE  statement.  In  the  OBU  system,  up  to  40 
query  statements  define  contact-to-track  association 
channels  with  10  more  for  track-to-track  association 
channels. 

The  partitioning  process  greatly  reduces  the  number 
of  tracks  that  must  be  compared  to  a  given  contact  re¬ 
port.  A  poorly  attributed  contact  (one  with  little  Informa¬ 
tion  other  than  a  position  and  time)  must  still  be 
compared  to  a  large  number  of  tracks:  lower  priority, 
more  general  channels  can  be  defined  for  processing 
them.  The  tracks  are  evaluated  each  time  a  new  contact 
is  placed  Into  a  channel  because  changes  to  a  track  since 
the  last  time  the  channel  received  data  for  processing 
may  change  Its  channel  qualifications.  While  a  contact  will 
be  placed  in  only  one  channel,  a  track  can  be  put  Into 
many  channels  and  therefore  updated  by  many  kinds  of 
contacts.  For  example,  consider  the  following  four  chan¬ 
nel  definitions: 

CHAN01:  [[CAT  EQ  NAV)  AND  NATO  AND  MED] 
CHAN05:  [[CAT  EQ  [NAV  OR  UNK]]  AND  NATO 
AND  MED) 

CHAN20:  [[CAT  EQ  [NAV  OR  UNK]]  AND  (NATO 
OR  [CC  EQ  UNK]]  AND  MED] 

CHAN30:  MED 
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Pre-Correlation 


Figure  5.  OBU  Correlator  Channelized  Data  Flow 


This  example  introduces  the  concept  of  priorities  to  the 
channelization  scheme;  channels  are  allocated  computer 
resources  according  to  their  channel  number,  with  chan¬ 
nel  1  having  highest  priority.  Contacts  are  processed  In 
the  highest  priority  channel  with  a  matching  query  state¬ 
ment.  In  the  following  discussion  of  routing  contacts  and 
selecting  candidate  tracks  for  the  example  channels,  It  is 
helpful  to  refer  to  Figure  6.  In  the  example,  all  contacts 
reported  as  NATO  Naval  platforms  In  Area  MED  will  be 
processed  at  highest  priority.  Reported  NATO  contacts, 
that  might  possibly  be  Naval  platforms  (those  for  which 
category  Is  not  reported)  are  processed  In  channel  5. 
These  contacts  are  processed  against  all  known  NATO 
Nava!  tracks  that  are  also  Included  In  channel  1  and 
against  all  NATO  Unknown  category  tracks.  Channel  20 
processes  contacts  with  unknown  country  (CC)  and 
either  unknown  or  Naval  categories  In  Area  MED  against 
Naval  or  unknown  category  tracks  from  NATO  or  un¬ 
known  countries  In  Area  MED.  Finally,  channel  30  proc¬ 
esses  contacts  not  qualifying  for  higher  priority  channels: 
in  the  example,  those  with  known  categories  other  than 
Naval  (e.g.,  Fishing,  Merchant,  etc.)  or  known  countries 
other  than  NATO  (e.g.,  UR)  in  Area  MED.  Note  that  a 
Greek  Naval  track  In  Area  MED  meets  all  four  channel 
definitions;  It  will  be  retrieved  and  processed  as  a  candi¬ 
date  against  contacts  In  all  four  channels. 


Channelization  allows  dissimilar  types  of  tracks  (air, 
land  and  wet)  to  coexist  In  the  same  data  base,  be  re¬ 
trieved  by  the  same  types  of  queries,  and  be  processed 
by  the  same  software,  without  Increasing  the  number  of 
contact-to-track  comparisons  per  contact  report  proc¬ 
essed.  It  allows  each  OSIS  site  to  customize  Its  process¬ 
ing  through  the  logical  definition  and  prioritization  of 


processing  channels.  Beyond  the  logical  definition  of  the 
channels.  It  provides  an  additional  degree  of  flexibility 
through  tailoring  the  algorithms.  The  correlation  algo¬ 
rithms  can  be  made  to  operate  differently  In  each  chan¬ 
nel.  The  three  levels  of  algorithms  (Unique  Attribute. 
ELINT  and  Spatial)  can  be  enabled  or  disabled  by  chan¬ 
nel.  Examples  are  to  disable  spatial  correlation  on  an  Air 
category  channel  or  to  disable  unique  attribute  correla¬ 
tion  In  an  ELINT  channel  (which  can  be  defined  by  sensor 
or  ELINT  Notations).  In  addition  to  enabling  or  disabling 
correlation  algorithms,  the  ELINT  and  spatial  algorithms 
have  sensitivity  adjustments  that  can  be  used  to  adjust 
the  Interdependent  rates  of  missed  correlation,  correct 
correlation,  wrong  correlation  and  ambiguous  correla¬ 
tion.  An  example  is  setting  lower  spatial  correlation 
thresholds  for  submarine  tracks  (which  are  Infrequently 
observed  and  relatively  small  in  number)  than  for  naval 
tracks  (which  are  more  frequently  observed  and  larger  In 
number).  Land  Mass  Avoidance,  an  Intensive  computa¬ 
tional  process,  can  also  be  enabled  or  disabled  on  a 
channel  basis,  enabling  It  for  areas  and  categories  of 
tracks  where  the  benefit  exceeds  the  cost.  All  of  these 
adjustments  can  be  made  at  each  site  as  a  part  of  cus¬ 
tomizing  the  fusion  process  to  the  environment. 


A  benefit  of  channelization  to  be  exploited  In  the  fu¬ 
ture  is  customizing  correlation  algorithms  on  a  channel 
basis.  For  example,  a  stationary  ELINT  target  tracking 
algorithm  for  correlation  of  land  based  radars  is  In  devel¬ 
opment.  This  algorithm  can  be  Inserted  into  channels 
designated  for  processing  of  land  category  contact  re¬ 
ports.  As  the  need  arises,  further  algorithm  specializa¬ 
tion  can  be  developed  and  inserted  Into  the  basic 
channelization  architecture. 
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Figure  6.  Channelization  Reduces  the  Number  of  Contact-to-Track  Comparisons 


Initiation  of  New  Tracks 


The  strategy  used  to  Initiate  new  tracks  Is  Important 
In  a  data  fusion  system  expected  to  maintain  a  large 
track  data  base.  The  number  of  contact-to-track  com¬ 
parisons  that  must  be  performed  Is  a  function  of  the 
number  of  tracks  In  the  data  base.  Even  when  “winnow¬ 
ing”  and  channelization  are  used  to  reduce  the  number  of 
candidate  tracks,  Indiscriminate  starting  of  tracks  may 
result  In  too  many  tracks  for  efficient  operation.  The  ana¬ 
lyst  must  deal  with  clutter  which  Is  also  a  function  of  the 
number  of  tracks  In  the  data  base.  Starting  tracks  that 
neither  grow,  nor  represent  Important  events,  Is  undesir¬ 
able  for  both  the  analyst  and  the  automatic  software.  On 
the  other  hand,  too  conservative  an  approach  to  track 
Initiation  results  In  lost  correlation  opportunities.  The 
OBU  architecture  avoids  the  classic  correlation  dilemma 
of  either  always  starting  single  point  tracks,  thereby  clut¬ 
tering  the  data  base;  or  starting  only  multiple  point 
tracks,  thus  losing  timeliness  and  early  correlation  oppor¬ 
tunity. 


Figure  4  Illustrates  the  combination  of  single-point 
and  multi-point  track  Initiation  functions  in  the  OBU  sys¬ 
tem.  Single-point  initiation  is  considered  for  every  con¬ 
tact  that  does  not  correlate  during  contact-to-track 
correlation.  New  tracks  are  Initiated  based  on  analyst  de¬ 
fined  “start  rules.”  A  set  of  rules  can  be  applied  to  un¬ 
correlated  contacts  In  each  of  the  40  contact-to-track 
correlation  channels.  These  rules  have  the  same  form  as 
the  channelization  queries  statements,  for  example: 


IF  [ELNOT  EQ  [C1234  OR  C5678]]  THEN 
START 

IF  [[NOSICID  NE  NULL]  OR  [CALLSIGN  NE 
NULL]]  THEN  START 

The  first  start  rule  Initiates  single  point  tracks  from 
any  uncorrelated  contact  having  one  of  the  two  specified 
ELINT  Notations.  The  second  rule  starts  a  track  for  an 
uncorrelated  contact  having  a  reported  NOSIC  ID  num¬ 
ber  or  Call  Sign.  This  technique  allows  timely  Initiation  of 
Important  tracks  without  cluttering  the  data  base  by 
starting  a  track  for  every  uncorrelated  contact. 

By  contrast,  multi-point  track  Initiation  Is  performed 
periodically  In  one  of  10  specially  defined  Initiation  chan¬ 
nels.  Uncorrelated  contacts  for  the  channel  (which  may 
be  defined  by  an  area,  a  time  window  and  other  contact 
attributes  like  ELINT  Notation)  are  gathered  and 
searched  for  potential  great  circle  route,  constant  veloc¬ 
ity  multi-point  tracks.  This  approach  will  be  most  effec¬ 
tive  In  open  ocean  areas  where  movement  tends  to  be  In 
straight  lines  for  a  period  of  time.  In  highly  congested 
areas,  or  regions  In  which  platforms  tend  to  maneuver,  It 
will  be  less  effective. 

Part  of  the  operational  tailoring  that  the  OBU  archi¬ 
tecture  allows  Is  Illustrated  by  Figure  7.  The  figure  shows 
a  port  with  a  ship  maintenance  area  (Area  1)  within  It.  It 
also  shows  an  area  off  the  coast  where  ships  normally 
conduct  exercises  (Area  4) .  Tracking  In  Area  1  on  either 
ELINT  or  spatial  information  Is  nearly  impossible  due  to 
the  erratic  nature  of  both  types  of  Information  (maneu¬ 
vering  In  congested  areas  can  make  spatial  behavior  ap¬ 
pear  random;  parametric  distortion  from  maintenance 
and  test  activity  can  cause  emitters  to  be  inconsistent 
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with  previous  observation) .  The  OBU  system  allows  defi¬ 
nition  of  a  set  of  “restricted  areas’  where  the  correlation 
algorithms  do  not  operate.  Contact  reports  In  these  ar¬ 
eas  are  not  discarded,  and  restricted  areas  can  be  en¬ 
abled  or  disabled  on  a  channel  by  channel  basis;  the 
channelization  scheme  can  allow  unique  attribute  Infor¬ 
mation  resulting  from  “port  watch’  to  be  correlated  \whlle 
not  attempting  ELINT  correlation  In  the  port  area. 


The  areas  defined  In  Figure  7  can  be  used  to  estab¬ 
lish  a  track  Initiation  strategy  for  the  part  of  the  world 
represented.  The  strategy  might  call  for  not  starting  new 
tracks  In  Area  1  because  tracking  In  this  area  is  too  diffi¬ 
cult.  Start  rules  might  dictate  starting  single  point  tracks 
in  Area  2  as  Indicators  that  a  deployment  Is  underway. 
Multi-point  Initiation  might  be  specified  for  contacts  In 
Area  3  that  were  not  associated  to  tracks  started  In  Area 
2.  The  multi-point  Initiation  channel  definition  might  also 
exclude  consideration  of  contacts  In  Area  4  because  It  Is 
known  that  significant  platform  density  and  maneuvering 
are  common.  In  addition  to  the  multi-point  Initiation  de¬ 
fined  for  Area  3,  specific  single  point  Initiation  rules  (such 
as  detection  of  a  unique  attribute  like  Call  Sign)  might 
also  be  defined. 


DEVELOPMENT  REALITIES 

Laboratory  versions  of  data  fusion  systems  exist  In 
synthetic  simulated  environments  that  can  be  controlled 
to  allow  concepts  and  algorithms  to  be  demonstrated  and 
proven.  When  developing  an  operational  data  fusion  sys¬ 
tem  for  deployment  Into  the  field,  there  are  additional 
practical  realities  which  must  be  addressed.  These  reali¬ 
ties  can  ccmpllcate  Implementaticn  of  the  system,  but  to 
Ignore  them  Is  to  risk  producing  an  algorithmically  correct 
but  operationally  unacceptable  system.  Four  of  these  re¬ 
alities  are  discussed  below. 

Fusion  Algorithms  Need  Supporting  Software 

The  amount  of  support  software  In  data  fusion  sys¬ 
tems  varies  greatly,  depending  on  the  environmental  fac¬ 
tors  discussed  in  the  first  section  of  this  paper.  The  OBU 
system  as  a  “dissimilar  source’,  ‘autonomous’,  “recep¬ 
tor/transmitter’,  ’undersampled’  correlation  system 
needs  more  supporting  software  than  systems  having 
simpler  environments.  In  Table  2,  the  software  in  the 
OBU  system  Is  allocated  to  functions  which  support  the 
overall  fusion  mission.  The  Environmental  Factors  column 
Indicates  which  OBU  functicns  are  Influenced  by  which 
environmental  factors.  The  total  software  represented 
by  the  functions  In  the  table  Is  approximately  600,000 
lines  of  code. 


Figure  7.  Definition  of  Operationally  Oriented  Regions  for  Initiation  of  Tracks 
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Table  2.  OBU  Software  Breakdown  By  Function 


Function 

Percent 

Of  OBU 
System 
Code 

Environmental 

Factors 

Correlation,  tracking 

7 

D,  A,  U 

Comms,  Input  msgs 

7 

D,  A 

Report  gen, 

output  msgs 

13 

D,  R/T 

Analysis  tools,  MMI 

17 

D,  U 

Data  management 

23 

D.  R/T,  OC 

System  setup,  control  33 

OC 

Environmental  Factors: 

D  Dissimilar  Source  Factor 

A  Autonomous  Location  Factor 

R/T  Receptor/Transmitter  Destination 

Factor 

U  Undersampled  Information  Factor 

OC  Operational  Considerations 


The  correlation  and  tracking  software  Is  Influenced 
by  the  dissimilar  types  of  source  Information,  the  autono¬ 
mous  location  of  the  system  away  from  the  sensors  (al¬ 
lowing  out-of-order  receipt  of  Information)  and  the 
undersampled  nature  of  the  data.  Even  with  these  fac¬ 
tors,  this  software  represents  less  than  ten  percent  of 
the  overall  code  in  the  OBU  system. 

The  dissimilar  source  and  autonomous  nature  of  the 
OBU  system  requires  significant  communications  soft¬ 
ware  to  receive  messages  from  the  several  circuits  pro¬ 
viding  data.  Input  message  handling  must  translate  a 
variety  of  message  formats  Into  the  homogeneously 
structured,  normalized  data  required  by  correlation  algo¬ 
rithms.  The  code  In  the  OBU  system  necessary  to  re¬ 
ceive  and  condition  the  data  prior  to  correlation  equals 
that  needed  to  perform  correlation  and  tracking. 

The  Receptor/Transmitter  nature  of  the  system  re¬ 
quires  report  generation,  message  construction  and 
message  transmission  capabilities.  The  fusion  of  dissimi¬ 
lar  security  classification  data  complicates  this  software. 
Because  of  the  need  to  provide  a  word  processing  capa¬ 
bility  to  generate  unformatted  output  reports,  the 
amount  of  code  to  support  the  generating  and  sending  of 
reports  and  messages  Is  nearly  double  that  needed  to 
perform  correlation  and  tracking. 

A  robust  set  of  analysis  tools  and  man-machine  In¬ 
terfaces  Is  necessary  because  the  dissimilar  sources 
(and  the  dissimilar  errors  they  make),  when  coupled  with 
the  undersampled  nature  of  the  data,  make  completely 
automatic  correlation  unachievable.  About  one-sixth  of 
the  software  In  the  system  contributes  to  support  of  ana¬ 
lyst  activities. 

Data  management  Is  impacted  by  the  fusion  of  dis¬ 
similar  security  classification  data  from  different  sources 
combined  with  the  external  reporting  nature  Inherent  In  a 
Receptor/Transmitter  system.  The  existence  of  multiple 
analysts  performing  interactive  analysis  and  generation  of 
output  reports  for  external  consumers  Is  an  operational 
consideration  which  adds  to  the  data  management  load. 
The  software  to  manage  updates  to,  retrievals  from,  and 


Integrity  of  the  resulting  distributed  data  base  constitutes 
nearly  one  quarter  of  the  system. 

System  setup  and  control  software  manages  the  lo¬ 
cal  network  architecture,  which  was  selected  largely 
based  on  operational  considerations:  throughput  and  tim¬ 
ing  requirements  forcing  separation  of  the  analyst  from 
the  automatic  data  flow,  and  system  availability  require¬ 
ments.  A  third  of  the  OBU  software  falls  Into  this  cate¬ 
gory,  which  also  Includes  software  necessary  to  control 
operation  of  the  correlation  process. 

Real  Systems  Must  Process  Real  Data 

Information  coming  Into  an  autonomous  data  fusion 
system  passes  through  a  potentially  distorting  filter  called 
a  message  format.  Even  though  formats  are  generally 
standardized,  reporting  conventions  can  add  nuances  to 
Individual  data  Items  that  must  be  considered.  A  system 
receiving  a  variety  of  message  formats  must  accommo¬ 
date  any  bias  between  sources  In  the  use  of  the  same  or 
similar  message  fields. 

Correlation  algorithms  based  on  unique  attribute 
data  are  particularly  sensitive  to  Imprecise  understanding 
and  Interpretation  of  specific  message  format  fields.  This 
sensitivity  Is  due  to  the  deterministic  matching  and  mis¬ 
matching  of  attribute  fields  Inherent  In  these  algorithms. 
Differing  conventions  between  sources  reporting  similar 
data  must  be  understood  and  accommodated  In  the 
processing. 

Parametric  data  can  be  distorted  through  the  mes¬ 
sage  formats  by  truncation.  An  ELINT  sensor  might  send 
out  contact  reports  In  several  different  formats.  Not  all 
formats  allow  the  same  number  of  digits  of  precision  in 
their  parametric  fields.  A  correlation  site  may  need  to 
choose  between  using  more  timely  Information  with  n  dig¬ 
its  of  accuracy  or  later  Information  with  n+1  digits  of  ac¬ 
curacy.  The  effect  on  the  mathematics  of  the  correlation 
process  Is  not  clear,  especially  If  n  and  n+1  digit  samples 
are  combined  together  In  statistical  calculations. 

Designers  often  rely  on  the  existence  of  data  ele¬ 
ments  defined  In  message  format  specifications.  In  fact, 
many  fields  may  be  reported  rarely  or  used  only  by  a  sub¬ 
set  of  the  sources.  Only  through  analysis  of  representa¬ 
tive  message  traffic  from  ail  the  potential  sources  can 
the  frequency  and  reliability  of  specific  data  elements  be 
determined.  Even  data  elements  which  are  specified  as 
“mandatory  fields”  In  message  format  specifications  are 
often  reported  as  the  field  was  filled,  meeting  the 

format  requirements,  but  no  Information  was  provided. 

The  need  to  obtain  “representative  real  data"  to 
support  the  design  and  testing  of  a  data  fusion  system  is 
compelling.  The  difficulty  in  so  doing  can  be  significant. 
Security  considerations,  data  variations  by  theater,  and 
the  Influence  of  speculations  and  myths  about  the  data 
are  factors.  Only  by  consulting  data  experts  In  the  field 
can  many  of  the  subtleties  of  the  data  be  discovered. 
Yet,  consultation  may  only  be  fruitful  after  prior  study  of 
sample  real  messages.  Analysts  in  the  field  are  generally 
more  familiar  with  observed  message  characteristics 
than  with  message  specifications. 

Another  aspect  of  real  data  Is  that  the  sensor  re¬ 
porting  world  is  dynamic.  Early  software  development 
must  protect  Itself  from  change  by  tightly  controlling  the 
assumptions  about  message  formats  and  using  artificial 
or  simulated  data  to  test  interfaces.  This  test  data  can 
be  effective  If  generated  from  an  understanding  obtained 
from  analysis  of  messages.  This  approach  alone  Is  often 
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not  enough  to  ensure  success.  Samples  of  real  mes¬ 
sages  must  be  periodically  analyzed  to  reaffirm  or  chal¬ 
lenge  earlier  assumptions.  Six  months  Is  a  recommended 
Interval  for  reexamining  the  content  of  the  messages  in 
the  field. 


Security  and  Recovery  are  Real  and  Difficult  Prob¬ 
lems 


In  a  speech  on  Navy  fusion  requirements  at  the  1987 
TrI-ServIce  Data  Fusion  Symposium,  RADM  K.  L. 
Carlsen  stated:  “These  security  barriers  are  a  fact  of 
life;  we  need  to  recognize  that  fact  In  our  thinking  about 
fusion  and  accommodate  It  early  In  the  design  of  fusion 
procedures  and  equipment”  (Reference  7).  The  OBU 
system  was  developed  with  this  goal  In  mind.  An  addi¬ 
tional  operational  constraint  was  the  need  to  recover 
from  hardware,  power  or  software  failures  without  losing 
either  the  accumulated  knowledge  represented  In  the 
track  data  base  or  the  contact  reports  currently  In  proc¬ 
ess. 


Security  and  recovery  are  not  usually  Important  re¬ 
quirements  in  the  development  of  test-bed  or  proof-of- 
concept  prototype  systems.  In  any  deployed  production 
fusion  system,  however,  these  Issues  must  be  success¬ 
fully  and  efficiently  addressed.  “Successfully”  addressing 
these  Issues  from  a  functional  point  of  view  Is  a  matter  of 
software  engineering  and  was  achieved  during  the  OBU 
development  In  a  straightforward  manner.  The  “effi¬ 
ciency”  aspect  was  more  difficult  and  took  considerable 
effort;  both  security  and  recovery  exact  a  high  price  In 
computer  resources.  The  challenge  for  any  large  scale 
data  fusion  system  development  Is  to  meet  required 
functionality  at  an  acceptable  cost  In  performance. 

Support  Data  Bases  Cannot  be  Ignored 

Many  data  fusion  systems  use  support  or  collateral 
data  bases  In  their  processing  or  to  assist  the  analyst. 
These  data  bases  can  be  as  varied  as  land  mass  avoid¬ 
ance,  shipping  lane  definitions,  platform  emitter  comple¬ 
ment  files,  or  emitter  mode  definitions.  These  data 
bases  must  be  defined  and  sized  early  In  the  develop¬ 
ment.  A  strategy  for  populating,  validating  and  providing 
life-cycle  support  for  these  data  bases  also  must  be  es¬ 
tablished.  Too  often,  only  sample  data  bases  to  support 
testing  are  developed;  the  development  assumes  that 
the  real  data  base  will  be  generated  by  operational 
personnel  after  the  system  Is  deployed.  As  a  result,  data 
fusion  systems  have  been  fielded  which  do  not  live  up  to 
their  potential  because  necessary  support  data  bases 
are  too  difficult  or  too  time  consuming  for  busy  site  ana¬ 
lysts  to  populate.  Just  as  important  Is  the  ability  to  main¬ 
tain  support  data  bases  once  In  the  field.  Without 
effective  tools,  modification  of  support  data  bases  may 
be  beyond  the  capability  or  inclination  of  a  data  fusion 
site.  Out  of  date  or  Incorrect  support  data  bases  could 
degrade  a  system’s  utility.  In  some  cases,  the  type  of 
Information  In  a  data  base  makes  It  necessary  to  obtain 
new  releases  periodically  from  a  central  location.  In  this 
situation,  a  method  of  Installing  new  data  base  releases 
which  accommodates  site  specific  customizations  must 
be  considered. 


DEVELOPMENT  CONSEQUENCES 

How  the  Development  Realities  discussed  above  af¬ 
fected  the  realization  of  the  OBU  system  Is  illustrated  in 
the  following  examples.  These  realities  not  only  Influ¬ 


enced  this  development  but  are  applicable  to  future  large 
scale  fusion  system  developments. 

System  Support  for  Data  Fusion 

A  major  consequence  of  the  environment  factors  on 
the  OBU  development  was  the  need  to  develop  consider¬ 
ably  more  software  than  Is  evident  from  the  fusion  algo¬ 
rithms.  Data  handling,  data  management,  analyst  tools 
and  system  control  became  major  aspects  of  the  devel¬ 
opment  effort.  At  times  during  the  development,  the 
computer  science  aspects  of  the  system  became  as  Im¬ 
portant  as  the  algorithmic  aspects.  Once  such  example 
f  ollovi/s . 

Accommodating  the  inherent  mismatch  between 
analyst  working  speeds  and  automatic  processing  speeds 
while  minimizing  mutual  Interference  proved  a  challenging 
software  task.  One  of  the  main  technical  dilemmas  was 
how  to  “lock"  tracks  to  avoid  simultaneous  update.  This 
is  a  classic  data  management  Issue,  but  In  this  case 
there  were  additional  considerations.  If  the  analyst  was 
allowed  to  lock  tracks  during  an  entire  period  of  analyzing 
and  manipulating  them,  the  automatic  software  would  fall 
behind  In  Its  work  because  a  correlation  channel  cannot 
run  unless  all  of  the  tracks  In  that  channel  are  available. 
Systems  with  lower  throughput  requirements  or  single 
analysts  can  often  afford  to  lock  a  track  or  the  entire 
data  base  while  an  analyst  manipulates  a  track:  the  OBU 
system  cannot. 

To  ensure  that  the  processing  resources  Invested  In 
performing  an  automatic  correlation  are  not  wasted,  the 
automatic  software  locks  all  tracks  In  a  channel  before 
beginning  correlation  In  that  channel.  Thus,  there  Is  a 
guarantee  that  a  successful  correlation  will  result  In  up¬ 
date  of  a  track,  and  that  the  track  did  not  change  since 
the  start  of  the  correlation  process.  The  analyst  must 
also  lock  tracks,  but  only  when  he  Is  ready  to  do  an  up¬ 
date.  During  active  periods,  the  automatic  software  can 
generate  a  high  demand  for  certain  tracks  and  many  re¬ 
sulting  requests  to  lock  these  tracks.  These  are  often  the 
same  tracks  needing  analyst  resolution  of  ambiguities 
The  analyst  cannot  wait  indefinitely  to  perform  an  up¬ 
date;  at  some  point  he  must  receive  a  “busy,  try  later” 
response.  However,  if  he  Is  unsuccessful  In  performing 
an  update  for  too  long,  the  automatic  software  may  up¬ 
date  the  same  track,  making  the  analyst’s  work  Invalid. 
The  solution  to  providing  timely  analyst  access  to  tracks 
without  Impeding  automatic  processing  is  to  institute  an 
analyst-biased  reservation  system.  In  this  way,  the  auto¬ 
matic  processing  can  complete  current  processing  but 
the  analyst  will  always  get  next  access  to  the  track  for 
update,  regardless  of  how  many  requests  for  the  track 
the  automatic  system  has  pending.  The  analyst  update 
will  only  lock  the  track  momentarily  and  then  the  auto¬ 
matic  system  can  continue. 

Another  aspect  of  synchronizing  the  analyst,  who 
works  slowly  compared  to  a  computer,  and  the  auto¬ 
matic  correlation,  which  works  quickly,  is  how  to  organize 
and  control  the  flow  of  data  from  the  computer  to  the 
analyst.  The  analyst  may  fall  behind,  especially  during 
heavy  data  load  periods.  Real  data  does  not  arrive  at  a 
uniform  rate;  there  are  distinct  peaks  and  valleys.  During 
peaks,  the  analyst  must  be  able  to  do  Important  work 
first  and  defer  less  Important  work.  The  OBU  system 
organizes  analyst  work  Into  "work  list"  queues.  Items  can 
be  put  into  queues  by  alert  filters  within  the  system  and 
by  the  correlator  generating  ambiguities.  To  allow  the 
analyst  selectivity  In  the  ordering  of  work,  each  of  the 
channels  in  the  correlator  is  capable  of  routinq  Its  ambi¬ 
guities  to  separate  work  queues.  By  planning’^the  setup 
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and  allocation  of  these  queues,  the  analyst  can  use  this 
division  of  \work  to  help  establish  priorities.  Within  a  given 
\work  queue,  a  two  line  summary  of  the  work  Item  allows 
an  Initial  assessment  of  the  urgency  of  the  particular 
work  Item. 

Large  systems  have  ancillary  functions  that  may  be 
considered  less  Important  during  development,  and 
therefore  receive  less  attention,  than  the  main  functions 
of  the  system.  An  ancillary  function  In  the  OBU  system  Is 
the  logging  of  Information  and  status.  Many  types  of  In¬ 
formation  are  logged:  Incoming  and  outgoing  messages, 
significant  events  associated  with  contacts,  security 
relevant  events,  significant  software  processing  events, 
and  system  alerts.  There  was  much  discussion  during  the 
development  of  the  system  of  what  data  Items  to  log  and 
where  to  log  them,  but  the  concept  of  how  the  results  of 
logging  were  to  be  used  was  not  fully  developed.  As  a 
result,  the  logging  Is  thorough,  but  the  extraction  of  In¬ 
formation  from  the  logs  to  perform  specific  analysis  Is 
sometimes  cumbersome .  An  effective  logging  system 
needs  to  emphasize  the  Information  to  be  extracted  from 
the  logs  and  the  goals  of  extracting  this  Information,  be¬ 
fore  becoming  Involved  with  the  detailed  definition  of  what 
is  to  be  logged. 


Real  Data 

Use  of  unique  attribute  data  for  correlation  can 
cause  correlation  anomalies  resulting  from  ambiguous  In¬ 
terpretation  of  message  format  fields.  For  example,  In 
the  Initial  OBU  system,  the  unique  attribute  correlation 
algorithm  deals  with  two  types  of  sensor  supplied  track 
numbers:  the  Precor  (Precorrelated)  Track  Number  and 
the  Raid  Number.  The  Precor  Track  Number  Is  taken  to 
Indicate  transmission  of  an  entire  track  at  one  time,  while 
the  Raid  Number  Indicates  an  update  to  a  previously  re¬ 
ported  track.  The  OBU  correlation  processing  is  different 
for  these  two  unique  attributes;  If  a  whole  track  Is  re¬ 
ceived,  there  Is  no  need  to  search  for  previous  points  be¬ 
longing  to  the  track  In  the  data  base.  Unfortunately, 
some  sensors  can  report  track  segments  (several  points 
for  a  track  In  a  single  message);  the  handling  of  a  track 
segment  encompasses  the  rules  established  for  both 
Precor  Track  Number  and  Raid  Number.  The  first  receipt 
of  a  segment  must  be  treated  like  a  Precor  Track  Num¬ 
ber  (put  all  the  points  In  the  data  base  as  a  new  track) , 
whereas  subsequent  segments  should  be  treated  like 
Raid  Numbers  (find  the  existing  track  and  extend  It) .  The 
message  processing  software  extracting  the  “track  num¬ 
ber"  has  no  reliable  way  of  assessing  which  kind  of  track 
number  is  being  received.  In  the  future  this  will  be  re¬ 
solved  by  combining  the  track  numbers  into  a  single 
unique  attribute,  relieving  the  message  handling  software 
from  making  an  unreliable  decision.  This  simplification  in 
message  Interpretation  must  be  accompanied  by  addi¬ 
tional  processing  in  the  correlation  software.  Upon  re¬ 
ceipt  of  a  track  number,  the  track  data  base  must  be 
queried  to  determine  existence  or  nonexistence  of  a 
track  matching  the  number  received. 

Another  example  deals  with  the  reporting  of  (Radio) 
Call  Sign,  which  Is  also  a  unique  attribute  In  the  OBU  sys¬ 
tem.  In  general,  a  given  Call  Sign  can  only  be  used  by  a 
single  platform.  However,  certain  conventions  have 
evolved  in  the  reporting  world  which  must  be  recognized. 
For  example,  the  Call  Sign  “NC”  means  “No  Call"  or  the 
equivalent  of  “Unknown."  A  system  correlating  on  Call 
Sign,  without  recognition  of  this  special  case,  will  attempt 
to  correlate  all  contacts  having  Call  Sign  NC  Into  a  single 
track,  generating  incorrect  results.  Similarly,  certain  Call 
Sign  values  are  used  as  a  special  convention  to  report 


collateral  Information  other  than  Radio  Call  Sign.  Knowl¬ 
edge  of  the  existence  and  values  of  these  special  con¬ 
ventions  must  be  built  Into  the  unique  attribute  correlation 
process. 

The  OBU  display  software  Initially  used  the  value  of 
the  reported  “Force  Code"  to  determine  the  symbols 
and  colors  to  display  for  a  track.  The  goal  of  this  coding 
Is  to  make  the  displays  more  readable  by  the  analyst. 
However,  analyzing  message  data  from  many  sensors  In¬ 
dicated  that  “Force  Code”  Is  rarely  reported,  even 
though  the  field  exists  In  the  message  formats.  As  a  re¬ 
sult,  appropriate  track  symbols  and  colors  were  not  gen¬ 
erated.  Most  sensors  prefer  to  report  country  code 
(flag)  and  category.  Force  code  will  be  derived  from  this 
Information  In  the  OBU  system  to  allow  correct  display  of 
colors  and  symbols. 

The  OBU  system  was  designed  with  the  recognition 
that  message  format  Interpretation  difficulties  are  un¬ 
avoidable,  especially  In  a  large  system  processing  many 
formats  from  many  sources.  An  ability  to  modify  mes¬ 
sage  parsing  rules  without  changing  software  has  helped 
the  system  adapt  to  the  real  data  encountered  In  the 
field. 

The  multl“Scurce  nature  of  the  OBU  system  requires 
Information  manipulation  prior  to  the  correlation  process. 
The  need  to  access  contact  report  Information  and  track 
information  by  software  throughout  the  system  led  to  the 
definition  of  a  standard  data  structure  for  contact  re¬ 
ports,  called  an  ICR  (Internal  Contact  Report),  and  one 
for  the  aggregate  track  Information,  called  a  track 
header.  These  data  structures  are  used  by  correlation 
software,  display  software,  data  management  software, 
report  generation  software  and  by  the  various  analyst 
tools.  A  single  format  for  dealing  with  Information,  inde¬ 
pendent  of  the  sensor  or  message  format,  is  convenient 
during  software  development.  It  results  in  easy  to  pro¬ 
duce  and  understand  code,  a  definite  goal  In  modern 
large-scale  systems.  It  also  appears  efficient  from  a 
performance  point  of  view,  because  once  Information  Is 
placed  Into  the  initial  structure,  further  data  transforma¬ 
tions  prior  to  use  are  not  required.  However,  due  to  the 
wide  variety  of  sensors  and  the  dissimilar  information 
they  report,  these  structures  are  significant  In  size  (ICR  = 
1024  bytes,  track  header  =  2560  bytes)  In  the  OBU  sys¬ 
tem.  They  are  often  very  sparsely  populated  with  Infor¬ 
mation  because  no  one  sensor  reports  (or  platform  has) 
all  of  the  attributes  allotted  space.  As  a  result,  the  sys¬ 
tem  often  stores  and  moves  largely  empty  data  struc¬ 
tures.  Simplicity  and  clarity  in  the  implementation 
extracts  a  subtle  penalty  In  storage  and  performance.  In 
the  area  of  performance,  the  gain  In  avoiding  transform¬ 
ing  the  data  structure  each  time  It  is  used  (e.g.,  by 
packing  and  unpacking)  is  at  least  partially  offset  by  the 
cost  of  moving  larger  structures.  From  a  simplicity  point 
of  view,  the  single  format  is  a  definite  advantage;  from  a 
performance  point  of  view,  the  advantage  depends  upon 
how  much  empty  space  must  be  moved  through  the  sys¬ 
tem  and  the  cost  of  this  movement.  This  Is  a  trade-off 
that  “dissimilar  source”  data  fusion  systems  must  make. 

Testing  to  Approximate  Real  Data 

Testing  Is  a  subject  related  to  the  characteristics  of 
real  data.  Initial  testing  Is  usually  performed  with  artifi¬ 
cially  constructed  data  that  can  conveniently  produce 
needed  scenarios.  Over  time,  the  test  data  must  evolve 
to  become  a  better  representation  of  the  real  world. 
Testing  of  a  data  fusion  system  requires  a  number  of 
tools.  “Ground  truth"  data  must  be  generated  by  simu¬ 
lation  and  used  to  test  algorithms.  During  early  Integra- 
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tion  of  the  OBU  system,  debugging  efforts  sometimes 
showed  that  the  software  worked  properly  but  the  test 
data  was  defective.  A  need  was  realized  for  additional 
tools  to  perform  quality  assurance  checks  on  simulation 
generated  test  data.  As  test  scenarios  reached  multiple 
thousands  of  contacts,  Independent  verification  of  the 
correctness  of  the  data  became  more  Important.  Using 
unproven  data  to  test  unproven  software  makes  problem 
resolution  less  timely.  An  engineering  challenge  recog¬ 
nized  early  In  the  OBU  development  was  management 
and  quality  assurance  of  the  test  data  to  achieve  variety 
without  losing  control. 


During  early  testing,  the  correlation  software  was 
separated  from  the  message  handling  software.  Simu¬ 
lated  contact  data  was  fed  directly  to  the  correlation 
software.  The  simulation  placed  the  data  Into  Internal 
Contact  Report  (ICR)  format.  After  a  time,  the  correla¬ 
tion  software  worked  properly  and  It  appeared  to  be  a 
straightforward  matter  to  combine  the  correlation  and 
message  handling  software.  Several  unexpected  practi¬ 
cal  details  made  this  transition  take  longer  than  ex¬ 
pected.  First,  the  data  simulation  software  now  had  to 
translate  the  contact  report  Information  Into  various 
message  formats.  A  way  had  to  be  found  to  store 
•ground  truth  track  number”  In  each  message  to  allow 
evaluation  of  correlation  correctness  by  test  tools.  This 
had  to  be  done  without  Influencing  the  message  handling 
or  correlation  processing  and  Invalidating  the  tests.  After 
this  detail  was  resolved,  a  second  Impediment  had  to  be 
overcome;  when  the  message  handling  software  parsed 
the  various  message  formats  Into  ICR  format  and  passed 
the  results  to  the  correlation  software,  the  correlation 
answers  were  sometimes  different  than  previously.  Dif¬ 
ferences  arose  from  how  the  simulation  software  popu¬ 
lated  the  fields  In  a  message,  from  how  the  message 
handling  software  parsed  the  message,  from  how  default 
values  for  missing  message  fields  were  handled,  and 
from  how  message  field  translations  were  performed.  A 
correlation  subsystem  that  can  develop  proper  tracks  is 
not  completely  checked  out  until  It  can  develop  tracks  In 
cooperation  with  a  message  handling  subsystem.  Much 
of  the  work  In  accomplishing  this  transition  lies  in  the  very 
low  level  practical  details  of  adapting  test  tools  and  de¬ 
termining  the  correct  construction  and  parsing  of  mes¬ 
sages.  Many  detailed  message  construction  or  parsing 
Issues  do  not  surface  until  the  message  handling  and  cor¬ 
relation  software  begin  to  work  together. 


To  thoroughly  test  and  evaluate  a  complex  correla¬ 
tion  system  like  the  OBU  system,  a  large  amount  of  test 
data  must  be  processed.  This  data  Is  generally  proc¬ 
essed  in  batch  runs  and  then  results  are  analyzed.  The 
underlying  philosophy  In  the  OBU  correlation  process  is  to 
rely  on  the  analyst  to  repair  Inconsistencies  In  the  data 
and  to  make  decisions  the  machine  cannot  make  be¬ 
cause  candidates  are  too  similar  to  distinguish.  “Ambi¬ 
guities”  deferred  to  the  analyst  tend  to  cascade  over 
time  If  not  resolved.  Runs  of  many  thousand  contacts 
without  working  off  ambiguities  can  result  In  a  large  per¬ 
centage  of  the  later  contacts  generating  ambiguities. 
This  occurs  because  earlier  ambiguities  have  not  been 
resolved;  If  left  In  the  data  base,  contacts  similar  to  ex¬ 
isting  tracks,  not  correlated  due  to  ambiguous  situations, 
tend  to  start  new  tracks.  The  result  Is  yet  more  tracks 
similar  to  earlier  tracks,  \Mth  the  potential  to  cause  future 
ambiguities.  On  the  other  hand,  providing  reasonable  and 
repeatable  manual  ambiguity  resolution  during  test  runs 
without  biasing  the  test  outcome  Is  a  challenge  In  a  soft¬ 
ware  development  environment. 


Security  and  Recovery 

Security  information  must  be  fused  by  a  multiple 
source  correlation  system  that  automatically  reports  re¬ 
sults  over  communications  circuits  (receptor/transmitter 
category  fusion  systems).  Classifications,  code  words 
and  releaslblllty  instructions  must  be  fused.  As  a  result, 
over  time,  the  information  In  a  track  will  evolve  to  the 
highest  levels  of  classification,  as  each  contact  added  to 
the  track  adds  security  Information.  The  track  Informa¬ 
tion  becomes  unavailable  for  reporting  to  consumers  with 
lower  level  clearances  and  the  ability  of  the  system  to 
effectively  discharge  Its  mission  can  be  diminished.  A 
major  dividing  line  between  consumer  clearance  levels  is 
whether  they  are  limited  to  receiving  GENSER  (General 
Service)  level  Information  only  or  whether  they  can  also 
receive  SCI  level  Information.  In  performing  multi-source 
fusion,  the  OBU  system  receives  both  categories  of  In¬ 
formation  and  must  support  both  categories  of  con¬ 
sumer.  In  order  to  meet  this  requirement,  the  system 
keeps  two  sets  of  books  on  the  track  data  elements 
which  are  used  for  Insertion  In  outgoing  event-by-event 
messages.  For  a  given  element,  there  Is  a  basic  track 
copy  that  Is  the  fused  result  of  all  contacts  In  the  track, 
and  a  GENSER-level  copy  that  Is  the  fused  result  of  only 
those  contacts  received  with  GENSER  security  classifi¬ 
cation.  Based  on  which  consumer  Is  to  receive  Informa¬ 
tion,  the  appropriate  copy  of  the  track  Is  used. 

Details  of  the  code  can  become  Important  In  fusing 
security  levels.  Consider  an  example  where  a  track  con¬ 
tains  a  radio  call  sign  of  ABCD.  This  call  sign  and  all  of 
the  remaining  data  In  the  GENSER  track  copy  came  from 
contact  reports  classified  “confidential”.  Another  con¬ 
tact  Is  received  that  Is  classified  “secret”  and  contains 
the  same  call  sign  value  “ABCD”.  If  the  software  does 
not  detect  that  the  call  sign  value  “ABCD”  already  exists 
In  the  track,  and  Instead  overwrites  the  existing  value 
“ABCD”  with  a  new  copy  of  the  same  value  “ABCD" ,  the 
security  level  of  the  track  must  be  upgraded  to  “secret” 
because  the  Information  was  derived  from  a  secret  mes¬ 
sage.  On  the  other  hand,  If  the  code  realizes  that  It  al¬ 
ready  has  the  call  sign  value  ABCD  In  the  track,  and  does 
not  update  the  Information,  the  level  of  classification  can 
remain  “confidential”. 

Security  and  data  base  recovery  requirements  serve 
to  exacerbate  the  complexity  of  the  throughput  perform¬ 
ance  estimation  effort.  When  added  to  the  Inefficiencies 
inherent  in  productivity  enhancing  high  level  languages 
and  vendor  supplied  system  software  services  like  file 
management,  system  throughput  and  processing  timeli¬ 
ness  can  no  longer  reliably  be  derived  from  algorithm 
complexity.  The  technique  of  determining  the  number  of 
mathematical  operations  or  lines  of  code  an  algorithm  re¬ 
quires  and  adding  15-25%  for  operating  system  overhead 
Is  no  longer  valid  in  modern  complex  systems;  If  relied 
upon  it  can  result  In  unexpected  performance  shortfalls. 
More  sophisticated  methods  of  analysis,  measurement 
and  finally  optimization  must  be  applied  In  a  system  that 
must  meet  particular  levels  of  timing  performance.  The 
OBU  software  was  subjected  to  significant  scrutiny  and 
rework  during  early  Integration  to  achieve  required  levels 
of  performance  In  the  face  of  stringent  security  and  re¬ 
covery  requirements. 

Either  multilevel  security  or  recovery  Implementa¬ 
tion,  If  not  carefully  designed,  can  reduce  a  computer 
system  to  25%  or  less  of  Its  expected  performance;  their 
combined  effect  can  leave  only  10%  of  the  computer  for 
application  processing.  With  performance  optimization, 
the  effect  of  either  can  be  reduced  to  the  point  where  the 
machine  will  perform  at  75%  of  expected  performance. 
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Thus,  the  compound  effect  of  multilevel  security  and  re¬ 
covery  can  still  reduce  the  application  level  performance 
of  the  computer  system  to  50%  of  the  performance  pos¬ 
sible  without  these  factors.  These  numbers  represent 
estimates  from  examining  the  OBU  system  and  several 
other  similar  large  systems.  Actual  determination  of  the 
computer  cost  of  security  and  recovery  Is  not  practical 
because  both  features  must  be  designed  Into  the  very 
basics  of  the  architecture  and  cannot  be  separately 
measured. 


Although  the  magnitude  of  performance  Impact  from 
security  and  recovery  Is  similar,  the  underlying  causes 
are  different.  Security  architectures  require  separation 
of  processing  of  data  with  different  security  levels.  This 
separation  mandates  movement  of  Information  within  the 
system  coupled  with  the  cleansing  (by  zeroing)  of  stor¬ 
age  space  previously  occupied  by  higher  classification  In¬ 
formation.  The  continual  movement  of  data  and  zeroing 
of  memory  effectively  empties  the  high  speed  cache 
memory  used  In  the  computers  which  host  the  majority  of 
data  fusion  systems.  These  computers  depend  on  suc¬ 
cessful  retention  of  frequently  used  information  In  cache 
memory  to  achieve  advertised  levels  of  performance. 
The  activities  required  to  build  secure  systems  work 
counter  to  modern  computer  architectures. 


The  performance  Impact  of  recovery  requirements 
is  from  a  different  cause.  As  a  part  of  data  base  recov¬ 
ery,  data  base  Integrity  must  be  considered.  A  data  base 
update  to  reflect  assignment  of  a  contact  to  a  track  re¬ 
quires  the  modification  and/or  creation  of  several  data 
base  entries  along  with  linkages  between  them.  If  a  sys¬ 
tem  failure  occurs  In  the  middle  of  a  string  of  such  trans¬ 
actions,  the  data  base  Is  left  In  an  Internally  Inconsistent 
state.  In  recovering  from  a  failure,  the  Incomplete  se¬ 
quence  of  transactions  must  either  be  completed  or 
rolled  back  to  the  beginning  to  start  over.  In  order  to  pro¬ 
vide  a  path  to  traverse  to  take  the  data  base  back  to  a 
consistent  state,  Information  must  be  saved  at  various 
points  during  processing.  This  information  must  be 
stored  (called  “journaling”)  on  a  disk  to  allow  Its  recovery 
after  a  failure.  In  a  distributed  data  base  system  with  a 
high  transaction  rate,  and  multiple  updaters  of  the  data, 
the  resource  cost  of  journaling  can  be  significant. 


Support  Databases 


The  OBU  Land  Mass  Avoidance  (LMA)  data  base 
can  be  used  to  highlight  the  Issues  of  developing  opera¬ 
tionally  useful  support  data  bases.  The  data  base  con¬ 
sists  of  two  parts,  a  land/water/shore  map  and  a  “points 
and  line  segment”  set  of  possible  routes.  Personnel  In 
the  field  will  have  better  knowledge  of  the  shipping  lanes 
traversed  by  platforms  of  interest  then  do  software  de¬ 
velopers,  but  personnel  In  the  field  may  not  have  time  to 
build  data  bases.  Therefore,  the  OBU  LMA  data  base 
was  computer  generated  by  connecting  points  derived 
from  standard  map  data  bases.  Obvious  shipping  lanes 
like  the  Panama  Canal  were  entered  manually.  The  two 
part  land  mass  avoidance  data  base  now  consists  of  sev¬ 
eral  hundred  thousand  points.  A  testing  program  was  in¬ 
itiated  to  detect  and  repair  anomalies  In  the  data  and 
algorithms.  Unfortunately,  it  was  found  that  testing  all 
paths  would  take  over  a  CPU~year .  A  random  testing 
program  was  then  Initiated.  After  several  months  of  fast¬ 
ing  and  repair,  the  data  base  anomaly  rate  was  reduced 
to  7  or  8  per  million  data  base  excursions.  The  errors 
continue  to  be  corrected,  but  at  a  testing  cost  much 
greater  than  anticipated. 


The  deployed  data  base  gives  the  site  an  Immediate 
ability  to  use  land  mass  avoidance,  but  because  it  was 
generated  from  maps,  It  does  not  account  for  the  subtle¬ 
ties  of  platform  movement  In  specific  theaters  of  activity. 
In  those  circumstances ,  the  system  provides  the  analyst 
a  tool  for  the  creation  of  additional  routes.  In  this  way, 
the  data  base  can  be  customized  to  the  needs  of  the 
site.  On  the  other  hand,  If  the  site  has  no  time  to  modify 
the  data  base,  there  Is  In  place  a  data  set  that  provides  a 
basic  level  of  functionality. 


FUTURE  DIRECTIONS  AND  POSSIBILITIES 
Directions 

The  current  deployed  version  of  the  OBU  system 
has  been  optimized  for  the  tracking  of  “wet”  naval  tar¬ 
gets.  An  on-going  development  effort  will  result  In  cus¬ 
tomization  for  efficient  correlation  of  aircraft  and  fixed 
land-based  targets.  The  architecture  described  In  this 
paper  will  remain.  Enhancement  will  be  in  the  areas  of 
system  responsiveness  to  analyst  command,  algorithm 
differentiation,  and  data  structure  definition. 

The  modifications  planned  for  aircraft  target  track¬ 
ing  are  largely  not  algorithmic  but  deal  with  Increasing 
system  responsiveness  to  the  analyst.  The  current 
unique  attribute  and  ELINT  algorithms  will  be  applied  to 
aircraft  platforms.  In  Its  Initial  orientation  toward  naval 
targets,  the  present  version  of  the  OBU  system  places 
emphasis  on  maintaining  Integrity  and  consistency  of 
tracks,  their  contacts,  and  the  updates.  This  is  done  at 
the  cost  of  decreased  analyst  responsiveness,  an  appro¬ 
priate  trade-off  In  an  environment  that  requires  mainte¬ 
nance  of  long  time-span,  lov/  update  rate  tracks,  in  the 
air  environment,  the  update  rate  of  individual  tracks  Is 
much  greater.  As  a  result,  the  system  must  be  more 
responsive  to  analyst  commands.  This  emphasis  on 
speed  Is  offset  by  the  fact  that  Individual  tracks  are  much 
shorter  In  duration,  so  long  term  track  Integrity  Is  not  as 
Important.  Trade-offs  that  were  made  in  favor  of  long 
term  track  integrity  are  being  revisited  for  air  tracking 
with  the  goal  of  making  the  system  more  responsive  to 
analyst  command,  indications  are  that  some  of  the  tech¬ 
niques  being  considered  for  air  tracks  will  also  apply  to 
naval  tracks,  thereby  Increasing  overall  system  respon¬ 
siveness.  Optimization  of  the  performance  of  commands 
that  air  analysts  use  frequently  with  prototype  air  proces¬ 
sors  Is  also  being  investigated.  These  optimizations  will 
benefit  all  analysts  on  the  system. 

The  advent  of  fixed-site  land  target  tracking  will  re¬ 
sult  In  Introduction  of  additional  correlation  algorithms 
Into  the  system.  These  algorithms  will  operate  within  the 
currently  defined  channelization  structure  on  specific 
channels  designated  to  process  land  data.  Unique  attrib¬ 
ute  algorithms  will  be  enhanced  to  include  land  target 
unique  attributes.  A  combined  ELINT/spatlal  correlation 
scoring  technique,  as  opposed  to  the  hierarchical  ELINT 
followed  by  spatial  scoring  scheme  used  currently,  Is  un¬ 
der  consideration. 

The  common  track  data  structure  may  need  modifi¬ 
cation  to  provide  a  specific  variation  for  land  tracks.  In¬ 
itial  Indications  are  that  combining  all  of  the  necessary 
information  Into  a  common  data  structure  may  make  the 
structure  too  large  for  efficient  movement  within  the  sys¬ 
tem.  The  land  correlation  problem  also  has  a  fundamen¬ 
tal  difference  from  the  naval  problem  that  can  be 
exploited:  a  large  number  of  targets  with  a  low  average 
update  rate  per  target.  This  Increases  the  probability  of 
retrieving  and  considering  a  large  number  of  tracks  for 
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each  contact.  A  dynamic  channelization  scheme  Is  being 
Investigated  to  use  the  fact  that  fixed  land  sites  don't 
move,  or  If  they  do,  not  very  far  or  very  fast.  Dynamic 
channelization  performs  the  geofeasibllity  test  as  part  of 
the  basic  channel  track  query,  not  retrieving  tracks  more 
than  a  certain  distance  from  the  contact. 

Possibilities 


Introduction  of  additional  algorithms  and  capabilities 
beyond  those  currently  planned  will  await  field  evaluation 
of  the  current  OBU  data  fusion  approach.  The  following 
paragraphs  speculate  on  potential  directions  for  the  fu¬ 
ture. 


Knowledge  based  system  concepts  and  techniques 
might  be  applied  to  analyst  ambiguity  resolution.  An  intel¬ 
ligent  analyst  work  station  could  be  made  capable  of 
monitoring  and  classifying  the  resolution  of  ambiguities  by 
an  analyst.  From  this  monitoring,  the  “ambiguity  resol¬ 
ver”  could  build  the  rules  and  use  patterns  for  various 
ambiguity  resolution  options  available  to  the  analyst.  The 
goal  would  be  to  collect  enough  understanding  of  ambigu¬ 
ity  resolution  patterns  to  begin  “suggesting”  to  the  ana¬ 
lyst,  upon  presentation  of  an  ambiguity,  the  actions 
typically  taken  In  the  past  (e.g.  “70%  of  the  time  this 
category  of  ambiguity  has  been  resolved  In  favor  of  the 
unique  attribute  data  In  the  FLINT  report”).  The  system 
could  be  trained  by  an  experienced  senior  analyst;  his 
pattern  of  decision  making  can  then  be  made  available  to 
a  less  experienced  analyst  who  works  a  different  shift. 

New  FLINT  approaches  like  DUFT.  may  be  consid¬ 
ered  If  they  are  demonstrated  to  offer  superior  perform¬ 
ance  to  TERFSA  at  acceptable  computer  resource  cost. 
Extension  of  TERESA  to  process  additional  FLINT  pa¬ 
rameters  and  to  handle  data  from  multiple  sensors  will 
probably  be  necessary  at  some  point.  Multiple  hypothesis 
algorithms  would  require  a  restructuring  of  Internal  data 
management  approaches  and  would  be  a  candidate  for 
Implementation  only  under  the  following  conditions:  1)  If 
the  ambiguity  rate  In  the  OBU  system  cannot  be  adjusted 
to  acceptable  levels,  2)  If  a  better  level  of  correlation 
performance  can  be  demonstrated,  and  3)  If  the  reser¬ 
vations  expressed  in  the  algorithm  section  of  this  paper 
can  be  overcome.  Performing  a  probabilistic  position  up¬ 
date  according  to  the  correlation  scores  of  the  several 
tracks  Involved  In  a  spatial  contact>to-track  ambiguity  is 
an  untried  heuristic  approach  that  falls  between  single 
and  multiple  hypothesis  techniques.  A  theoretical  basis 
for  such  an  approach  would  need  to  be  developed  before 
It  could  be  seriously  considered. 

It  wll!  be  necessary  to  address  methods  of  Integrat¬ 
ing  the  capabilities  of  prototype  systems  currently  opera¬ 
tional  at  the  OSIS  sites  to  produce  a  single  analyst  view 
of  the  ocean  surveillance  picture.  Functional  Interfacing 
to  prototypes  conforming  to  specified  Interface  stan¬ 
dards  Is  possible.  The  Interfaced  systems  would  be  sub¬ 
ject  to  scrutiny  to  ensure  operational  and  security 
integrity.  As  key  features  of  today's  prototypes  are  de¬ 
termined  to  be  necessary  In  performing  the  ocean  sur¬ 
veillance  function,  they  can  be  Introduced  Into  the  full 
scale  engineering  development  process.  Incorporation  of 
these  features  Into  the  OBU  system  as  the  site  data  fu¬ 
sion  hub  will  likely  be  paralleled  by  the  appearance  of  new 
prototype  systems,  addressing  the  continually  changing 
nature  of  the  correlation  environment  and  the  fusion  site 
personnel's  desire  to  experiment  with  new  functions. 

The  OBU  system  bases  correlation  decisions  on 
unique  attribute,  FLINT  and  location  Information.  Since 


each  consists  of  dissimilar  types  of  Information,  during 
the  design  of  the  OBU  fusion  approach  It  was  felt  that 
there  was  little  theoretical  basis  to  perform  automatic 
correlation  based  upon  a  combined  “goodness  of  fit"  en¬ 
compassing  all  of  the  Information.  The  result  was  the  Im¬ 
plementation  of  the  hierarchical  set  of  correlation 
algorithms  discussed  In  this  paper.  In  this  hierarchy,  a 
correlation  decision  (Including  the  decision  to  Inform  the 
analyst  of  an  ambiguity)  can  be  reached  at  any  of  the 
three  levels.  Lower  levels  of  the  hierarchy  are  not  con¬ 
sulted  if  the  computations  at  any  level  produces  a  suffi¬ 
ciently  good  correlation.  Similarly,  once  a  lower  level  is 
reached  In  the  correlation  hierarchy.  Information  from 
upper  levels  is  not  used  In  attempting  to  make  the  corre¬ 
lation.  The  result  Is  less  than  optimal  use  of  the  available 
intelligence  data  In  reaching  decisions.  A  decision  algo¬ 
rithm  which  makes  use  of  all  three  types  of  Information  to 
arrive  at  an  optimal  correlation  decision  might  be  devel¬ 
oped.  In  particular,  techniques  for  computing  a  combined 
“goodness-of-fit"  for  FLINT  and  spatial  characteristics 
are  a  definite  possibility.  Additionally,  provisions  for  site 
defined  unique  attribute  “confidence  factors"  In  specific 
situations  (e.g.,  sensor  1  makes  a  correct  NOSIC  ID  as¬ 
signment  60%  of  the  time  for  ELNOT  xxxxx)  might  be 
Investigated.  Currently  all  unique  attribute  data  is  treated 
as  being  true  -  Independent  of  the  source,  correlation 
situation  or  availability  of  supporting  information.  Applying 
probabilistic  confidence  measures  to  unique  attribute  as¬ 
signment  could  allow  Integration  of  the  unique  attribute 
level  of  the  correlation  hierarchy  with  the  parametric  and 
spatial  attribute  levels  of  correlation. 

Correlation  Engines 

Many  of  the  ideas  listed  as  “possibilities"  above  as 
well  as  new  algorithms  coming  from  the  research  com¬ 
munity  may  well  Improve  the  data  fusion  capabilities  of 
the  OBU  system.  The  basic  system  architecture  will  sup¬ 
port  significant  growth  In  the  hardware,  software  and  al¬ 
gorithm  dimensions.  On  the  other  hand,  It  may  prove 
difficult  to  provide  the  machine  power  necessary  to  ac¬ 
commodate  Increased  Input  data  rates  that  might  occur 
In  the  Intermediate  term  and  also  accommodate  newer 
correlation  approaches  (which  always  seem  to  require 
more  computation)  without  degrading  the  timeliness 
characteristics  of  the  system.  In  addition,  as  input  rates 
increase,  the  number  of  ambiguities  the  analyst  must 
handle  may  also  tend  to  Increase.  More  thorough  treat¬ 
ment  of  all  the  Information  available  or  use  of  multiple  hy¬ 
pothesis  algorithms  might  be  able  to  decrease  ambiguity 
rates.  Once  again,  more  processing  power  will  be  neces¬ 
sary. 

There  are  two  ways  to  address  the  potential  need 
for  more  processing  power  In  the  future.  One  is  to  wait 
for  the  newer  and  faster  general  purpose  computers 
which  appear  every  few  years.  The  second  is  to  consider 
development  of  a  correlation  “engine".  Such  an  engine 
could  be  built  from  ASIC  (Application  Specific  Integrated 
Circuit)  technology  or  programmable  high  performance 
microcomputer  chips.  By  allowing  such  a  device  read¬ 
only  access  to  the  OBU  track  database  and  treating  Its 
output  as  a  recommendation  to  the  OBU  computer  to 
perform  a  track  update  (add  contact  A  to  track  B),  such 
a  device  could  be  incorporated  Into  the  OBU  system  se¬ 
curity  and  recovery  architectures.  This  engine  could  pro¬ 
vide  added  computing  power  optimized  for  correlation 
processing  and  would  allow  consideration  of  more  com¬ 
plex  correlation  algorithms  while  performing  more  con- 
tact-to-track  comparisons  per  unit  of  time  and  achieving 
greater  throughput.  This  approach  could  also  provide  a 
means  of  adding  more  sophisticated  algorithms  to  the 
OBU  system  while  maintaining  the  strengths  of  the  cur- 
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rent  system.  Among  these  strengths  are  the  operational 
considerations  accommodated  In  the  basic  architecture: 
timeliness  and  throughput  performance,  robustness  of 
recovery  and  database  Integrity,  strict  and  accreditable 
security  features,  extensive  communication  connectivity, 
a  wealth  of  support  tools  provided  to  the  analyst,  and  a 
distributed  architecture  which  supports  multiple  analysts 
without  Interfering  with  the  speed  of  automatic  process¬ 
ing. 

External  Interfaces 

As  the  OBU  system  takes  Its  place  as  the  hub  of  the 
Navy’s  ocean  surveillance  function,  Increased  emphasis 
will  be  placed  on  formalizing  a  distributed  data  base  con¬ 
cept  capable  of  handling  both  ashore  and  afloat  systems 
and  accommodating  prototypes.  This  effort  will  be  re¬ 
quired  to  handle  data  flow  to  and  from  afloat  tactical 
systems  while  maintaining  Internal  data  base  Integrity. 
Considerable  experience  with  the  current  system,  real 
data  and  better  understanding  of  site  specific  needs  will 
be  required  to  permit  definition  of  a  structure  that  meets 
the  performance  and  Information  needs  of  both  tactical 
and  Intelligence  users. 
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REAL-TIME  AUTOMATED  TRACK  MANAGEMENT 


Susan  J.  Feldman 


HUGHES  AIRCRAFT  COMPANY,  INC. 
P.O.  Box  3310 
Fullerton,  CA  92634 


The  Advanced  Combat  Direction  System  (ACDS)  Block  1,  intended  for  all  United  States  Navy  non-Aegis 
cruisers  and  carriers,  brings  to  the  fleet  significant  new  capabilities  for  automated,  integrated,  real¬ 
time  track  management.  At  the  sensor  source  level,  making  use  of  government  furnished  information  and 
operator-entered  libraries,  ACDS  interprets  and  evaluates  data  from  both  cooperative  and  non  cooperative 
tracks.  At  a  higher  level,  ACDS  automatically  associates  tracks  and  then  fuses  classification  data 
derived  from  a  variety  of  sources  both  organic  and  external  to  the  ship.  The  resulting  track 
identification  drives  threat  evaluation,  is  displayed  to  Combat  Information  Center  tactical  decision¬ 
makers,  and  is  transmitted  over  tactical  data  links  (including  TADIL  J)  to  other  Battle  Force  members. 

Sitting  at  the  hub  of  a  distributed  information  system,  ACDS  must  interface  both  with  existing  systems 
and  with  new  systems,  many  of  which  are  in  concurrent  development.  Each  of  them  has  its  peculiar  point  of 
view  for  measuring  and  evaluating  tracks  and  its  own  taxonomic  language  for  classifying  the  tracks  based 
on  observed  characteristics  and  user  requirements.  Clearly  interoperability  has  been  a  major  concern  in 
the  development  of  ACDS  Block  1  and  many  related  practical  problems  have  had  to  be  solved  including 
taxonomies,  libraries,  the  man-machine  interface,  and  software  implementation.  Additionally,  new 
algorithms  have  had  to  be  developed  to  deliver  required  performance  in  the  areas  of  track  correlation  and 
Electronic  Support  Measures  (ESM)  data  integration.  This  paper  addresses  the  ACDS  approach  to  solving 
these  problems  and  provides  performance  modeling  results. 
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INTRODUCTION 

The  commander  of  a  Battle  Force  or  a  Navy 
surface  vessel  in  the  next  decade  will  be  faced 
with  a  busy  threat  environment  consisting  of 
many  tracks,  arriving  quickly  and  distributed 
across  a  vast  surveillance  region.  Modern 
shipboard  computers  and  Link  16  make  possible 
the  processing  and  communication  of  the  high 
volume  of  associated  data.  But  without  automated 
real-time  track  management  and  decision  support 
processing  to  reduce  the  data  into  usable 
information  the  commander  and  his  Combat 
Information  Center  staff  will  be  overwhelmed. 
The  Advanced  Combat  Direction  System  (ACDS) 
Block  1  is  being  developed  to  meet  this  need 
for  automated  track  management  .  ACDS  Block  1 
will  be  installed  on  U.S.  Navy  carriers  and 
other  large  non-AEGIS  surface  ships  during  the 
1990 *s.  Block  I's  very  ambitious  requirements, 
including  the  automated  fusion  of  all  available 
information  for  use  in  the  real-time  tactical 
environment,  will  make  it  the  Navy's 
operational  state-of-the-art. 


P.URP.QSE 

The  purpose  of  this  paper  is  to  provide  an 
overview  of  real-time  automated  track  management 
and  data  fusion  as  they  will  be  implemented  in 
ACDS  Block  1.  The  paper  will  address 
requirements,  design  and  techniques  used, 
performance,  and  related  issues.  The  reader  is 
invited  to  apply  to  the  Naval  Ocean  Systems 
Command  (NOSC)  for  details  beyond  the 
unclassified  security  level  of  this  paper. 


ACDS  BLOCK  ■l  .  OVERVIEW 

ACDS  Block  1  will  replace  the  Naval 
Tactical  Data  System  (NTDS)  on  carriers  and  non- 
AEGIS  cruisers  (and  potentially  other  large 
surface  ships)  with  new  software  that  will  give 
the  system  expanded  surveillance,  command 
support,  and  weapon  control  plus  multi-link 
capability,  ACDS  is  being  designed  specifically 
for  TADIL  J,  the  protocol  for  the  new  tactical 
data  link.  Link  16. 


ACDS  Block  1  is  required  to  combine  real¬ 
time  and  non-real-  time  track  data  from  all 
available  data  sources  so  that  identified  tracks 
will  be  available  for  immediate  display  and 
threat  processing.  This  applies  to  all  tracks, 
both  cooperative  and  non-cooperative,  in  the 
Navy  dispersed  battle  force  surveillance  volume, 
illustrated  in  Figure  1. 

The  ACDS  track  management  function 
correlates  remote  and  local  tracks  from  many 
sources,  including  radar  and  Electronic  Support 
Measures  (ESM) ,  and  automatically  identifies  the 
combined  track.  Red,  blue,  and  neutral  tracks 
may  be  classified  at  a  level  of  detail  that  is 
useful  for  command  decisions  and  is  consistent 
with  STANAG  1241,  the  basis  for  the  TADIL  J 
taxonomy.  Data  link  protocols  for  ID  precedence 
and  difference  resolution  are  followed.  The 
operator's  principal  role  in  track  management  is 
to  control  the  system  and  act  as  an  arbitrator 
when  conflict  situations  arise. 


Figure  1:  ACDS  Block  1  Operating  Environment 


ACDS  is  hosted  in  standard  Navy  computers 
(AN/UYK-43)  using  the  Navy  standard  high  order 
computer  language  (CMS-2)  within  a  tactical 
real-time  operational  frame.  These  requirements 
and  the  extension  of  the  surveillance  region  to 
be  included  within  the  ACDS  track  stores  dictate 
an  implementation  approach  that  makes  efficient 
use  of  computer  memory  and  processing. 
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RTiOrK  1  ARCHITECTURE 

An  overview  of  the  Block  1  architecture  is 
illustrated  in  Figure  2.  ACDS  interfaces 
directly  with  the  ship’s  sensor /trackers, 
weapons,  and  also  with  other  systenis  whose 
principal  concern  is  non-real-time  track 
processing.  ACDS  interfaces  with  other  battle 
force  units,  via  the  Command  and  Control 
Processor  (C2P) ,  over  Links  4, 11, and  16. 
Support  is  provided  for  approximately  25 
operator  modes,  depending  on  the  platform. 


Figure  2:  ACDS  Block  1  Architecture 


The  Block  1  program  is  housed  in  two 
AN/UYK-43  computers,  one  the  Track  Management 
Processor  (TMP)  and  the  other  the  Decision 
Support  Processor  (DSP) .  The  TMP  is  essentially 
a  data  fusion  box;  the  major  technical 
challenges  for  the  TMP  were:  development  and 
validation  of  data  fusion  algorithms  that 
predictably  meet  quantifiable  performance 
requirements;  and  (b)  software  techniques  to 
manage  the  computational  load.  The  DSP  is 
essentially  an  expert  system  driven  by 
operator-definable  rules;  the  major  technical 
challenges  for  the  DSP  have  been:  (a)  devising  a 
man-machine  interface  that  puts  the  operator  in 
control  of  the  automaticity  and  presents  him 
with  complex  information  in  an  understandable 
form:  and  (b)  software  techniques  for  applying 
large  numbers  of  rules  to  large  numbers  of 
tracks  in  real-time.  (In  this  way  ACDS  Block  1 
has  incorporated  into  standard  Navy  language  and 
computers  many  of  the  features  commonly 
associated  with  artificial  intelligence  (AI)  and 
specialized  AI  machines  and  languages.)  The  ACDS 
data  base  consists  of  dynamic  track  data  and 
tactical  status  files,  the  operational  doctrine 
rule-base,  and  government  furnished  libraries 
and  maps.  The  TMP  produces  and  updates  the 
track  record  data  which  is  then  used  by  the  DSP 
for  decision-making  and  display. 


TRACK-TO-TRACK  CORRELATION 

ACDS  Block  1  is  charged  with  creating  a 
non-redundant  track  file  out  of  the  data 
produced  by  all  its  input  sources,  which  usually 
arrives  in  the  form  of  tracks  created  by 
external  systems  rather  than  raw  sensor  data. 
Quantitative  performance  requirements  for 
correlation  have  been  established.  For  the  case 
of  remote-local  (radar-based)  tracks,  the 
requirements  are  expressed  as  %  of  dual 
designations  (DD) ,  i.e.  redundant  data  link 
track  reports,  and  %  of  false  correlations  (FC) 


for  tracked  objects  spaced  at  specified 
statistical  distances  (functions  of  sensor 
standard  deviation.)  These  correspond  to  Type  1 
and  Type  2  error  limitations.  The  DD  and  FC 
requirements  are  sufficiently  ambitious  that 
they  cannot  be  met  simultaneously  when 
correlation/  decorrelation  decisions  are  based 
on  such  simple  criteria  as  relationship  of 
position  deltas  to  inner  and  outer  bins. 
Historically  emphasis  has  been  placed  on 
maintaining  a  low  DD  rate,  since  this  decreases 
the  traffic  burden  on  the  data  link,  even  at  the 
expense  of  a  higher  FC  rate.  Although  false 
correlations  often  resolve  themselves  in  time  as 
the  falsely  correlated  objects  separate  in 
position,  an  erroneous  identification  may  be 
produced  by  mingling  the  wrong  evidence.  Thus, 
in  order  both  to  meet  quantitative  correlation 
requirements  and  to  increase  identification 
quality,  development  of  a  sound  technique  for 
track-to-track  correlation  in  a  busy  environment 
has  been  a  prime  Block  1  concern. 

Correlation  Algorithms 

A  common  technique  for  track-to-track 
correlation  was  developed  which  is  used  for 
remote/local  radar  tracks,  ESM/radar  tracks,  and 
ESM  line-of-bearing  correlation.  In  this 
sequential,  multiple  hypotheses  scheme, 
candidates  for  correlation  with  a  new  track  are 
selected  using  a  gross  correlation  test.  For 
each  candidate  a  correlation  score  is 
established  and  revised  by  filtering  as  updates 
are  received  from  the  sensors.  A  correlation  is 
declared  when  the  score  rises  above  a 
correlation  threshold,  (and  if  certain  conflict 
tests  are  passed,  including  ID  conflict)  and  is 
broken  when  the  score  falls  below  a 
decorrelation  threshold.  Candidates  are 
eliminated  only  when  the  filtered  score  drops 
below  a  purge  threshold.  Until  then  the  scoring 
process  continues  even  if  another  candidate  has 
passed  the  correlation  threshold. 

Trade-off  studies  for  choices  for 
threshold  values  and  filter  gain,  which  affect 
the  Type  1  and  Type  2  error  rates  and  the  time 
required  (number  of  updates)  to  reach  a 
decision,  were  performed.  It  was  determined 
that  Block  1  quantitative  requirements  for  DD 
and  FC  could  be  met  through  this  scheme  if  the 
tracks  being  compared  had  unbiased  position 
estimates  and  reasonably  accurate  covariance 
data  were  available.  Most  of  the 
trackers/systems  with  which  ACDS  Block  1  will 
interface,  however,  were  developed  to  interface 
with  earlier  combat  direction  systems  that  did 
not  require  accurate  track  covariances.  So, 
although  the  systems  generally  develop  and 
maintain  covariances  as  part  of  their  tracking 
process,  error  data  is  not  passed  over  the 
interface.  This  is  true  even  for  Link  11  track 
quality  (TQ)  .  Consequently  Block  1  contains 
algorithms  for  estimating  covariances.  Also, 
since  relative  biases  must  be  expected  among 
tracks  produced  by  distributed  systems  and 
hardware.  Block  1  includes  algorithms  for  bias 
estimation  and  correction.  These  include 
estimation  of  inter-sensor  bias,  as  between  the 
ownship  ESM  and  radar  trackers,  and  gridlock 
correction. 

The  automatic  ongoing  gridlock  correction 
process  accounts  for  remote/local  misalignment 
in  x,  y,  azimuth,  and  rangescale  which  are 
induced  by  biases  in  navigation  system 


299 


estimation  of  ownship  position  and  radar 
tracking  biases.  The  process  corrects  for 
misalignment  with  the  Link  11  Gridlock  Reference 
Unit  and  for  misalignment  with  each  Link  16 
participant  (Link  16  has  no  GRU.)  A  least  mean 
squares  batch  processed  gridlock  corrrection 
algorithm  is  used  which  requires  less  computer 
processing, (CPU)  time,  with  comparable  bias 
reduction  performance,  than  a  Kalman  filter 
approach . 

Combined  DSI/Gridlock  Performance 

DSI  remote/local  correlation  performance 
and  gridlock  performance  are  interdependent . 
With  large  biases  remaining  between  remote  and 
local  track  reports  for  the  same  tracked  object, 
no  correlation  or  false  correlations  will  occur 
since  the  decision  statistics  are  taken  from  an 
unbiased  model  of  reality.  The  gridlock  pads 
are  estimated  using  accumulated  differences 
between  track  data  from  correlated  remote  and 
local  reports,  thus  when  incorrect  correlation 
decisions  are  taken,  the  gridlock  pads  will  not 
be  accurate. 

To  evaluate  combined  DSI  correlation  and 
gridlock  performance  and  refine  the  algorithms, 
a  system  model  was  developed  that  simulates 
sensors,  data  link,  and  Block  1  track 
management .  A  scenario  was  created  to  stress 
both  correlation  and  gridlock,  illustrated  in 
Figure  3,  that  consists  of  20  pairs  of  targets 
arranged  in  a  spiral  and  converging  on  the 
Gridlock  Reference  Unit  (GRU) .  Each  target  pair 
is  represented  by  a  •  and  the  members  of  each 
pair  are  spaced  in  accordance  with  a  specified 
performance  spacing  requirement.  Early  in  the 
scenario  only  some  of  the  objects  are  mutually 
detectable  by  the  GRU  and  the  other 
participating  unit  (PU) . 
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Figure  3:  DSI/Gridlock  Scenario 


Figure  4  illustrates  gridlock  quality 
achieved  compared  with  required  quality  over  the 
scenario  life  for  one  model  execution. 
Substantial,  but  not  unrealistic,  biases  were 
applied  to  the  GRU*s  radar  azimuth  measurements 
and  to  the  GRU*s  estimate  of  its  own  position  in 
(x, y).  The  graph  illustrates  excellent  gridlock 
quality  over  the  scenario  life.  The  quality 
metric  is  a  normalized  measure  of  the  position 
match  of  remote/local  mutual  tracks  after 
gridlock  correction  pads  are  applied. 


Figure  4:Gridlock  Quality 


Figure  5  illustrates  average  (Monte  Carlo) 
correlation  performance  over  the  scenario  life 
in  terms  of  %  of  dual  designations  (DD)  ,  i.e. 
redundant  tracks  and  %  of  false  correlations 
(FC) .  Four  scenario  cases  are  shown,  identical 
except  for  spacing  between  target  pair  members. 
Biases  were  applied  to  the  GRU  as  described 
above.  The  first  case  was  run  using  the 
required  spacing  for  the  specified  standard 
case;  required  performance  for  this  case,  in 
terms  of  maximum  per  cent ages  of  DD  and  FC,  is 
shown  as  a  horizontal  line.  The  other  cases 
illustrate  performance  when  spacing  between  pair 
members  is  reduced  to  three-quarters,  one-half, 
and  one-quarter  of  the  required  standard 
spacing.  Note  that  the  simulated  correlation 
outperforms  the  requirement,  and  is  still  quite 
good  even  when  objects  are  quite  closely  spaced. 
This  allows  considerable  margin  for  at-sea 
performance  with  real  sensors  where  a  more 
random  mix  of  spacing  is  to  be  expected  and 
clues,  such  as  ID  and  IFF  responses,  will  be 
available  to  aid  correlation. 


Figure  5:  Correlation  Performance  with  Gridlock 
Bias  Correction 

ESM  Tracking 

The  ESM  SSI  distinguishes  among:  (1) 
intercepts,  which  are  detected  emission 
measurements  with  an  estimated  line  of  bearing 
(LOB) ;  (2)  emitters,  the  physical  devices  that 

are  emitting;  and  (3)  platforms,  the  ships  or 
aircraft  on  which  the  emitters  are  located. 
Correlation  of  two  or  more  intercepts  with 
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tiatching  waveform  measurements  produces  an 
emitter  (x,y)  position  and  velocity  estimate  and 
an  area  of  probability  (AOP) .  Spatial 
correlation  of  two  or  more  emitters  yields  an 
estimated  position  and  classification  of  the 
platform  carrying  them.  The  quality  of  the  data 
upon  which  the  ESM  SSI  operates  varies 
considerably  depending  on  the  source.  For  the 
most  part,  however,  the  bearing  measurement  data 
will  be  imprecise,  produced  by  shipboard  or 
aircraft  ESM  sensors. 

The  ESM  SSI  is  required  to  perform 
"automated  triangulation."  It  was  determined 
early  on,  however,  that  merely  finding  the 
intersection  point  of  two  bearing  lines 
associated  with  non-simultaneous  detections  by 
different  sensors  of  (presumably)  the  same 
emitter  was  not  sufficient  to  meet  the  larger 
goal  of  classifying  non-cooperative  radar  tracks 
using  associated  emission  data.  It  became  clear 
that  an  ESM  tracker  was  required  that  would 
produce  unbiased  position  estimates,  good 
covariance  estimates,  very  good  velocity 
estimates,  and  recognize  and  adjust  for 
maneuvers.  The  velocity  estimates  are  used  both 
for  extrapolation  and  to  reduce  classification 
ambiguity  when  radar-based  velocity  is  not 
available.  The  figure  below  shows  simulated 
tracker  performance,  using  line  of  bearing 
reports  from  two  sensors,  in  following  an  object 
through  2  turns  executed  at  3  degrees  per 
second.  Initially,  while  the  tracker  settles  and 
the  geometry  is  poor  (the  bearing  lines  at  first 
are  nearly  parallel),  the  (x,y)  positions  are 
imprecise  but  unbiased,  the  range  estimate  is 
poor,  while  the  bearing  estimate  is  good.  Later 
the  tracked  position  follows  the  true  position 
closely.  Of  more  importance  to  correlation,  the 
tracker  produces  very  good  estimates  of  90% 
error  elipses  and  velocity - 


MnnP.T.  S  TAXONOMY 

Data  in  the  track  stores,  and  corresponding 
symbology,  are  organized  in  terms  of  a  "Model  5" 
taxonomy  defined  for  use  within  ACDS  Block  1  in 
order  to  minimize  loss  and  distortion  of  data 
from  tactical  and  intelligence  community 
sources,  and  maximize  compatibility  with  TADIL  J 
and  the  latest  global  data  fusion  systems  such 
as  the  Flag  Data  Display  System  (FDDS)  and  the 
Afloat  Correlation  System  (ACS) .  Generally,  the 
external  systems  and  libraries  with  which  ACDS 
Block  1  will  interface  don't  agree  on  taxonomy 
structure,  parameters,  or  admissible  parameter 


values.  (Existing  tactical  identification 
systems  used  today  by  NATO  and  US  tactical 
forces  are  defined  in  the  messages  used  on  TADIL 
A  (Link  11)  and  TADIL  B,  with  JINTACCS  and  NTISA 
enforcing  interoperability  for  systems  using 
these  data  links.  The  intelligence  and  over- 
the-horizon  targetting  (OTH-T)  communities  use 
an  entirely  different  system  based  on  character 
oriented  messages,  standardized  in  the  RAINFORM 
Maritime  Reporting  System  message  set.) 

The  objective  of  identification  processing 
is  to, fill  in  the  blanks  on  a  track  by  supplying 
a  suitable  value  for  each  parameter  in  an 
identification  taxonomy.  The  ideal  taxonomy 
would  be  universally  useful,  providing  a 
complete  and  unambiguous  set  of  choices  for  each 
parameter.  The  structure  would  include^  a 
hierarchical  description  capability,  ranging 
from  detailed  to  general  where  the  most  detailed 
level  would  still  be  consistent  with  the  kinds 
of  discrimination  achievable  with  the  given 
sensors.  The  structure  would  also  lend  itself 
tb  maintaining,  independent  of  a  stable 
description,  judgments  about  temporary  track 
characteristics  such  as  organizational 
associations  and  threat  potential.  And,  in 
fact,  the  simplified  models  of  reality  that  are 
used  in  algorithmic  research  make  these  sorts  of 
assumptions  about  the  taxonomic  language  behind 
the  algorithms. 

Unfortunately,  since  reality  is  not  neatly 
structured  and  our  sensors  each  see  reality  from 
a  different  perspective,  none  of  the  taxonomies 
commonly  in  use  today  satisfies  all  the  above 
criteria.  Nor  is  it  likely  that  a  perfect 
taxonomy  could  be  devised.  As  an  example 
illustrating  the  problem,  consider  the  Russian 
"Bear"  aircraft.  The  name  "Bear"  calls  to  mind 
"Bear  bomber",  but  actually  designates  both  the 
TU-95  and  the  TU-142  and,  depending  on  the  type 
modification,  indicates  either  a  bomber, 
reconnaissance  or  ASW  aircraft: 

TU-95  Bear  A  Bomber 

TU-95  Bear  D  Reconnaissance 

TU-142  Bear  F  ASW 

The  ACDS  Block  1,  or  Model  5,  taxonomy 
structure  was  developed  both  to  maximize 
commonality  with  STANAG  1241  and  TADIL  J  and  to 
overcome  well-known  limitations  in  the  Link  11 
(Model  4)  taxonomy.  The  resulting  Model  5 
taxonomy  provides  an  integrated  track 
identification  that  can  accommodate  data  from 
all  sources,  including  ELINT  and  rainform- 
derived  classifications.  It  supports  the 
Composite  Warfare  Command  (CWC)  concept  by 
providing  information  for  all  categories  of 
tracks.  The  structure  differentiates  clearly 
between  reference  points  and  tracks,  and 
describes  all  tracks  in  a  similar  fashion 
irrespective  of  category  or  data  source,  while 
minimizing  the  interaction  of  fields.  This 
makes  consistent  MMI  for  data  entry  and  review 
possible,  no  matter  what  identification  data  is 
displayed,  which  is  particularly  important  for 
review  and  creation  of  doctrine  rules  for  ID, 
threat-handling,  and  display  filtering. 

The  taxonomy  is  organized  around  three 
semi-independent  axes: 

(1)  Description  answers  the  question 
What  is  it?  This  is  an  object  invariant.  The 
level  of  detail  ranges  from  category  through 
platform,  type,  and  type  modification.  Even  the 
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unit  —  side  number  or  hull  number  —  may  be 
discernible  by  some  sensors,  such  as  the  human 
eye,  so  a  place  is  provided  in  the  taxonomy  for 
maintaining  this  data. 

(2)  Threat  Potential  answers  the 
question  What  is  it  doing?  This  includes 
Identity  (Friend,  Neutral,  Hostile,  etc.)  and 
activity  and  status  parameters. 

(3)  Organization  tells  to  whom  the 
track  belongs.  It  includes  Nationality  and 
Alliance  parameters  as  well  as  the  service 
organizational  structure. 

Additionally  indicators  of  special 
considerations,  such  as  exercises,  and  data 
sources,  such  as  PPLI  (precise  position  locator 
message  transmitted  by  Link  16  participants)  or 
IFF,  are  maintained  internally  for  the  purpose 
of  modifying  identification  or  confidence 
values. 

The  Model  5  taxonomy  structure  is  based  on 
that  of  TADIL  J  (implicit  in  the  message  set) 
which  comes  close  to  meeting  the  desired 
taxonomy  features.  Some  additional,  i.e.  not 
for  transmission,  parameters  have  been  defined 
for  ownship  use  only.  Admissible  values  will  be 
a  superset  of  parameter  values  either  defined  in 
the  data  link  message  standards  or  in  use  by  the 
other  systems  with  which  ACDS  Block  1 
interfaces . 

By  contrast,  the  Link  11  standard 
identification  structure  used  by  current  "Model 
4"  tactical  data  systems  such  as  AEGIS  C&D-ADS 
and  NTDS  Model  4  is  considerably  more 
complicated.  Vehicular  track  data,  EW  track 
data,  data  about  link  participants.  Anti- 
Submarine  Warfare  (ASW) -related  data,  data  from 
intelligence  sources,  and  data  about  controlled 
aircraft  must  each  be  stored  in  a  different 
Model  4  format.  This  data-dependent  structure 
is  almost  unworkable  in  a  data-fusing  system 
where  it  is  possible,  for  example,  to  have  one 
track  file  record  representing  an  aircraft  under 
the  control  of  an  air  controller,  participating 
on  the  data  link,  responding  to  IFF 
interrogation,  and  detectable  by  the  SLQ-32  ESM 
sensor.  Model  4  track  description  fields  depend 
on  the  assigned  identity  so  that  a  change  from 
Friend  to  Hostile  can  require  reformatting  the 
data.  The  admissible  values  for  Primary 
Identification  (PRI  ID)  and  identification 
amplification  (ID  AMP) ,  which  in  combination 
determine  the  base  symbology  displayed  on  the 
console  screen,  contain  a  mix  of  activity  and 
description  values;  a  change  in  one  of  these 
values  can  require  regenerating  the  displayed 
symbol  for  a  track.  Finally,  Model  4  is  neither 
as  complete  nor  as  detailed  as  Model  5  will  be. 
For  example,  intelligence  information  is  not 
implemented  or  integrated  with  surveillance 
data,  there  are  no  land  special  points,  and  much 
ESM-derived  classification  data  can  be 
transmitted  only  in  the  form  of  emitter  numbers. 
Figure  7  illustrates  the  increased 
classification  capability  possible  due  to  the 
change  in  taxonomy  from  Model  4  to  Model  5. 


Figure  7:  Model  4  and  Model  5  Taxonomies 


TDF.NTTFTCATTON  SYSTEM  OVERVIEW 

The  ACDS  design  shown  in  Figure  8  provides 
for  a  multi-level  identification  process.  The 
two-tiered  integration  process  has  available  to 
it  a  variety  of  stored  data  including:  a  priori 
presence  probabilities,  libraries, 
speed/altitude  relationships,  IFF  code  look-ups, 
and  operator-modifiable  ID  doctrine  rules. 


Figure  8:  MSID  Overview 

At  the  initial  fusion  level.  Similar  Source 
Integration  (SSI)  functions  combine  sensor  data 
received  from  sets  of  independent  sensor  data 
processors  with  similar  characteristics,  and 
derive  classification  alternatives  and 
confidences. 

In  the  ESM  SSI,  tracks  are  classified  in 
one  of  two  ways:  (1)  either  by  applying  a 
Bayesian  technique  together  with  knowledge  about 
joint  distributions  of  emission  parameters  (per 
the  ESM  libraries)  and  measurement  noise;  or  (2) 
by  comparison  with  an  Emitter  Recognition  File 
which  stores  electronic  signatures  of  previously 
seen  or  anticipated  objects.  The  Emitter 
Recognition  File  is  useful  for  storing  Friendly 
Electronic  Order  of  Battle  data  and  for  rapid 
classification  of  intermittently  emitting 
radars . 

In  the  Radar/IFF  SSI,  IFF  interrogation 
responses  from  cooperative  objects  are 
interpreted  into  taxonomy  parameter  values  by 
accessing  an  operator-modifiable  IFF  look-up 
library.  (Reception  of  codes  not  in  the  library 
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results  in  prompts  to  the  operator  to  add  to  the 
library.)  Confidence  is  assigned  to  an  IFF- 
tagged  track  based  on  clues  such  as  combination 
of  mode-responses  received,  match  with  air  plan, 
compatibility  of  object  kinematics  with  derived 
classification, et  cetera#  i.e.  the  same  sorts  of 
clues  customarily  used  by  an  operator  in 
assigning  identification  based  on  IFF. 
Classification  of  non-cooperative  tracks  to  the 
category  level,  and  sometimes  to  the  platform 
level,  is  performed  by  comparing  observed  track 
kinematics  with  stored  speed/altitude  profiles. 

Also  at  the  initial  integration  level, 
other  sources,  which  concentrate  on  non-real 
time  processing,  contribute  identification 
products  which  must  be  incorporated  into  the 
final  identification.  These  include,  for 
example,  acoustic  classifications  produced  by 
the  CV-ASWM.  ACDS  preprocesses  these  products  so 
that  they  also  arrive  at  MSID  in  the  form  of 
alternatives  with  confidences.  The 

preprocessing  includes  taxonomy  normalization 
and  confidence  estimation.  Some  loss  of  data 
and  performance  degradation  occurs  which  might 
be  avoided  if  the  interfaces  were  designed  to 
make  such  processing  unnecessary. 

At  the  higher  level,  after  correlation  by 
the  Dissimilar  Source  Integration  (DSI) 
function,  the  Multi-Source  Identification  (MSID) 
function  combines  classification  and  confidence 
data  to  arrive  at  fused  alternatives  and  a 
"best"  identification  for  the  track.  A  Dempster- 
Shafer  approach,  selected  as  most  appropriate 
for  combining  the  classifications  thus  produced 
through  distributed  processing,  is  augmented 
with  knowledge  about  known  relationships  such  as 
weapon-launch  platform,  basing,  and  air  corridor 
relationships.  The  result  is  a  list  of  multi¬ 
level  alternatives,  with  confidences.  At  this 
point  a  ranking  algorithm  is  invoked,  which 
takes  into  account  both  confidence  and  level  of 
detail,  to  determine  which  is  the  best 
identification  for  the  track.  For  example, 
suppose  the  alternatives  for  the  same  track 
include : 

AIR,  FIGHTER,  MIG  23  Confidence 

=  0.45 

AIR,  BOMBER  Confidence  =  0.49 

AIR  Confidence  =  1,0 

Here  the  highest  confidence  alternative  — 
just  AIR  —  yields  the  least  information.  Which 
of  the  other  alternatives  is  best  is  not  at  all 
obvious.  The  "best"  identification  thus 

determined,  after  consistency  and  conflict 
checking,  will  be  entered  in  the  track  stores, 
displayed,  and  transmitted  over  the  data  link. 

The  ID  (Friend,  Hostile,  .  .  .)  value  is 

assigned  by  applying  data  link  protocols  and 
operator-modifiable  doctrine  rules.  Because  ID 
is  really  a  political,  not  a  permanent, 
characteristic  of  an  object,  ACDS  uses  automated 
doctrine  rules  for  flexibility  in  adjusting  ID 
assignment  to  changing  rules  of  engagement. 


THE  OPERATOR'S  ROLE 

Ideally  an  automated  identification  system 
would  free  the  operator  from  dealing  with 
individual  tracks,  but  in  reality  conflicts  and 
ambiguities  will  occur.  Furthermore,  operator 
data  entry  will  be  required  for  data  from 


systems  lacking  a  digital  interface,  such  as 
voice  reports.  ACDS  alerts  the  operator,  as 
required  by  data  link  protocols,  to  process  data 
link  conflicts  and  provides  displays  and  menus 
for  selection  of  responses.  (Based  on  the  AEGIS 
experience,  such  conflicts  are  to  be  expected 
when  new,  automated  systems  and  the  earlier 
manual  systems  operate  together  in  the  Battle 
Force.)  The  authorized  operator  may  also 
override  an  automatically  supplied 
identification  on  any  track;  the  system  flags 
the  track  so  that  the  automated  process  will  not 
subsequently  override  the  operator.  Similarly, 
the  operator  may  correlate  two  tracks;  track 
data  comparison  displays  are  provided  him. 
ACDS  Block  1  also  maintains  SSI  data  which  the 
operator  may  access  to  make  a  decision  to 
decorrelate  a  source  from  an  already  correlated 
track. 

When  the  operator  is  acting  in  his  data 
entry  capacity,  the  identification  algorithms 
require  that  his  input  be  formatted  as 
alternative (s)  with  confidence (s) .  The  MMI  has 
been  designed  with  menu  choices  of  taxonomy 
values  and  confidence  values  to  foster  human- 
algorithm  interoperability,  preventing^  spelling 
and  other  legality  errors.  For  example,  the 
operator  may  cursor  to  CATEGORY  and  select  Air, 
Fighter,  Probable  then  cursor  to  TYPE  and  select 
MIG  23,  Possible.  If  he  had  already  entered  US 
as  NATIONALITY,  MIG  23  would  not  appear  on  his 
menu  while  F-4  would. 

Controlling  Automaticitv 

The  automatic  process  is  driven  by 
algorithms,  doctrine  statements,  and  libraries. 
The  operator  tunes  DSI  correlation  algorithms  by 
selecting  from  system  settings  that  correspond 
to  "normal",  "slow  but  sure",  and  "quick  and 
dirty";  the  effect  of  these  controls  is  to 
modify  thresholds  and  filter  gain.  Declaration 
of  track  description  or  nationality  results  when 
the  identification  algorithm-produced  confidence 
exceeds  a  declaration  threshold.  The  operator 
tunes  the  identification  algorithms  through 
"certainty  controls",  selecting  the  standard, 
default,  setting  or  "more  certainty  required"  or 
"less  certainty  required";  the  effect  of  these 
controls  is  to  modify  the  declaration 
thresholds. 

In  order  to  tailor  automatic  ID-setting  to 
operational  policy,  the  operator  may  create  or 
modify  automatic  doctrine  statements.  For 
example,  "If  AIR  &  USSR  &  rangeOOO  NM,  then  set 
ID=SUSPECT."  Doctrine  statements,  or 
automatically  fired  if-then  rules,  have  been 
successfully  employed  by  the  AEGIS  ships;  care 
has  been  taken  to  preserve  interoperability  when 
implementing  them  in  ACDS  Block  1.  The  "if" 
clause  contains  multiple  tests  of  track  record 
contents,  such  as  ID  and  kinematics,  joined  by 
logical  connectors  "and",  "or",  and  "not".  The 
"then"  clause  is  an  action  to  be  performed  by 
the  system  whenever  the  "if"  test  is  passed. 
Track  management  actions  include  changing  the  ID 
or  platform  value  assigned  to  a  track,  and  IFF 
interrogation.  (The  principal  use  of  automated 
doctrine  in  Block  1,  however,  is  for  threat 
ranking  and  engagement  decisions.)  ACDS  Block  1 
provides  powerful  console-specific  display 
filtering  capabilities  because  its  increased 
surveillance  volume  and  track  file  capacity 
contain  far  more  data  than  can  be  displayed  at 
any  one  console.  Display  filters  and  alert 
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criteria  (as  well  as  threat  evaluation/  weapon 
assignment  rules)  are  also  entered  into  the 
system  as  doctrine  statements. 

Automated  ■D.octx i.ne 

As  illustrated  in  Figure  9  the  operator  may 
predefine  doctrine  statements  and  store  them 
for  activation  either  singly  or  as  a  set.  Thus 
in  effect  he  can  reprogram  the  system  to  include 
standard  operating  procedures. 


Figure  9:  Use  of  Automated  Doctrine 


S0FTWARE_TECHNTQI1ES 

The  ACDS  Block  1  program  will  be 
manipulating  vast  quantities  of  data  to  meet 
real-time  tactical  needs.  In  order  to  handle 
the  associated  requirements  for  computation, 
rule-firing,  search,  and  display,  a  new  way  of 
organizing  the  software  program  v;as  required. 
Block  1  features  many  techniques  which,  though 
proven  in  commercial  applications,  are 
innovative  for  Navy  software  programs.  Figure  10 
illustrates  the  approach. 


The  program  is  organized  as  data-driven, 
object-oriented  code  and  contains  an  Object 
Management  System  (OMS) .  Whenever  possible 
"adaptable  parameters"  (input  data)  are  used 
instead  of  hard-v;ired  data.  This  reduces  the 
numbers  of  lines  of  code  that  need  to  be  written 
and  tested  and  makes  it  easier  to  adapt  the 
program  to  future  needs. For  example,  display 
formats  are  defined  in  terms  of  track  store 
objects  and  adaptable  parameters  and  thus  can 
easily  be  reformatted.  Large  portions  of  the 
libraries  which  are  stored  on  disk  are  also 
represented  in  active  memory  as  objects  with 
pointers  to  their  disk  location;  this  speeds  up 
the  process  of  reading  from  disk  and  eliminates 
many  reads  altogether.  The  "if"  conditions  for 


the  automated  doctrine  rules  may  reference 
zones;  for  example,  "if  an  aircraft  exits  a 
commercial  air  corridor (a  zone),  then  .  .  . 

The  zones  also  are  defined  as  bit-map  objects, 
rather  than  in  terms  of  equations  of  lines  and 
circles;  location  of  a  track  object  inside  or 
outside  the  zone  can  then  easily  be  determined 
by  set  logic  operations  rather  instead  of  by 
expensive  computation.  Track  stores  and 
libraries  are  constructed  as  relational 
databases  and  the  program  features  a  high-speed 
query  process.  The  doctrine  rule  base  contains 
both  active  and  inactive  rules;  active  rules  are 
combined  via  a  dynamic  logic  tree. 


The  ACDS  algorithms  are  expected  to 
properly  associate  tracks  and  then  produce  valid 
identification  results  speedily  to  support  real¬ 
time  tactical  decisions.  But  decision  quality 
and  the  quantity  of  tracks  that  can  be  correctly 
correlated  and  identified  are  affected  adversely 
by  problems  and  issues  beyond  the  internal  ACDS 
Block  1  design,  notably: 

(1)  Interfacing  systems  and  libraries 
each  speaking  a  different  language,  where  no 
language  is  complete  or  perfectly  structured, 
resulting  in  loss  of  data,  ambiguity,  and  extra 
processing; 

(2)  Systems  trading  identification 
products  resulting  in  the  loss  of  statistical 
independence ; 

(3)  Model  4  manual  systems  (or  other 
automated  systems  using  different  algorithms, 
including  PC  programs)  ,  operating  in  conjunction 
with  Model  5  automatic  systems,  producing 
conflicting  identification  or  correlation 
results; 

(4)  Costly  vjorkarounds  and  estimation 
procedures  when  good  data  and  error  estimates 
maintained  internally  by  subsystems  are  not 
provided  across  interfaces  to  the  data  fusor. 

The  proofs  or  justifications  for  algorithms 
that  combine  classification  and  identification 
data  from  multiple  sources  are  based  on 
assumptions  about  the  data  which  is  to  be 
combined.  Ideally  the  data  provided  to  the  ACDS 
Block  1 ‘  MSID  process  would  fit  those 
assumptions.  That  is,  the  data:  (1)  would  have 
been  derived  independently  by  the  several 
sources;  (2)  would  be  equally  valid  for  each 
source;  (3)  would  be  reported  in  a  common 
language;  (4)  would  include  all  plausible 
classification  alternatives;  and  (5)  would 
assign  confidences  that  have  the  same  meaning  to 
those  alternatives.  (For  example,  if  one  system 
Uses  zero  confidence  to  denote  certain 
impossibility,  another  system  should  not  use 
zero  confidence  to  denote  lack  of  knowledge.) 

The  reality  of  the  situation  faced  by  MSID 
is  far  from  ideal.  As  indicated  in  Figure  11, 
many  of  the  systems  which  provide  data  and 
products  to  ACDS  also  trade  data  and  products 
v;ith  each  other.  ACDS  can  neither  know  nor 
control  whether  the  data  it  processes  from  an 
external  source  represents  new,  independent 
evidence.  Furthermore,  the  existence  of 
identification  fields;  or  even  identification 
confidence  fields,  in  an  IDS  message  is  no 
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guarantee  that  the  content  of  the  message 
represents  identification/confidence  data  of  the 
sort  ACDS  expects  to  process.  As  a  result,  ACDS 
must  cope  by  preprocessing  the  data  not  only  to 
normalize  the  taxonomy  values  as  previously 
discussed,  but  also  to  assign  or  scale 
confidences  and  attempt  to  fill  in  the  blanks. 


The  other  systems  in  the  web,  many  of  which 
are  still  in  early  development,  will  also  have 
to  deal  with  this  problem.  SPAWARS  has  been 
reexamining  shipboard  system  architecture  in 
order  to  consolidate  and  eliminate  overlapping 
functions.  NATO  working  groups  continue  efforts 
at  standardization,  not  only  of  the  taxonomy, 
but  also  of  the  format  for  interchange  of 
statistical  confidence  related  to  identification 
information.  In  the  long-term  the  solution  must 
be  to  address  data  integrity  as  an  issue  and 
include  safeguarding  requirements  when  new 
systems  and  architectures  are  devised. 


CONCLUSION 

ACDS  Block  1  was  designed  to  meet  the 
surface  Navy’s  need  for  data  fusion  in  the 
present  and  future  operating  environment.  The 
program  is  responsive  to  fleet  requirements  for 
increased  range  and  track  capacity  and  improved 
integration  and  automation.  It  produces  a 
comprehensive  tactical  picture  using  high 
performance  algorithms  and  assists  the  operator 
in  evaluating  that  picture  through  operator 
specified  automated  doctrine.  But  ACDS  Block  1 
is  only  one  element  of  the  total  shipboard  and 
battle  force  system.  Its  performance  depends 
on  the  quality  of  its  input  systems  and  on  the 
architecture  that  links  it  with  other  systems 
that  are  also  performing  data  fusion.  It  remains 
to  be  seen  whether  the  combat  system 
architecture  of  the  future  will  promote  good 
fusion  performance  or  adversely  .affect  it. 

As  Admiral  C.A.H.  Trost,  Chief  of  Naval 
Operations,  said,  *'It  is  for  consideration 
whether  we  will  have  the  discipline  and  the 
foresight  to  build  one  fused  system,  with  no 
matter  how  many  peripheral  components,  or  have 
the  equivalent  of  a  couple  of  hundred  individual 
personal  computers  that  cannot  talk  to  each 
other. " 

The  Advanced  Combat  Direction  System  Block 
1  is  the  first  major  step  in  building  that  fused 
system. 
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Abstract:  There  is  an  alarming  lack  of  technology  available  for  automated  systems  to  perform  the 
sophisticated  high  level  reasoning  necessary  to  react  quickly  to  reports  pertaining  to  the  location  and 
posture  of  opposing  forces.  At  the  most  fundamental  level,  it  is  essential  to  be  able  to  rapidly  process  an 
arbitrary  map  coordinate  by  associating  it  with  the  spatial  feature(s)  nearest  at  hand,  especially  when 
feature  boundaries  vary  in  time.  The  US  Army  Center  for  Signals  Warfare  has  conceptualized  a  theory  of 
front  end  spatial  perception  which  provides  a  solution  to  this  problem.  The  theory  has  produced  three  new 
computer  algorithms  called  loci -reduced  spotioX  discriminatiotiy  aunulus-based  inclusion  testing  for 
multiply -connected  sets,  and  topographical  contour  betweenness  testing.  The  first  two  techniques  exploit 
data  structures  called  the  equidistance  loci  set  and  the  inner  annulus  as  respective  criteria  for  nearness  and 
closed-contour  inclusion.  Building  upon  loci  reduction,  the  last  technique  extends  the  first  technique  into 
the  third  dimension  to  reason  about  a  coordinate's  directional  gradient.  On  maps  of  real  world  bounda^ 
complexity,  the  theory  has  facilitated  the  automation  of  the  following  spatial  reasoning  computations  in 
real-time:  nearest  (time-varying)  feature  to  an  arbitrary  coordinate;  distance  of  a  feature  from  an  arbitrary 
coordinate;  relative  direction  of  a  feature  from  an  arbitrary  coordinate;  a  decision  about  whether  an 
arbitrary  coordinate  is  inside  or  outside  a  feature  characterized  by  a  multiply-connected  closed  contour;  the 
two  elevation  contours  between  which  an  arbitrary  coordinate  lies;  the  local  directional  gradient  of  an 
arbitrary  coordinate.  When  combined  in  a  package,  loci  reduction,  annulus-based  inclusion  testing,  and 
tdpographical  contour  betweenness  testing  provide  a  powerful  toolset  of  utility  functions  for  command  and 
control  map  reasoning  processes. 
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INTRODUCTION 

The  process  of  reasoning  about  the  location  and 
posture  of  opposing  forces  is  of  paramount  importance  to 
the  success  of  friendly  force  command  and  control 
operations.  Actions  taken  by  either  side  are  a  function  of 
how  force  elements  are  positioned  with  respect  to  both 
natural  and  manmade  spatial  features.  At  the 
perceptual  level,  an  automated  battlefield  map 
reasoning  tool  must  be  capable  of  spatially  associating  a 
coordinate  with  the  geographic  features  and  force 
elements  nearest  at  hand.  The  process  should  also  be 
able  to  perform  inclusion  testing  on  features  which  are 
contained  within  other  features,  and  should  be  able  to 
understand  the  implications  of  local  topography  upon 
communications  and  maneuvers.  Such  a  low  level 
perceptual  capability  is  required  across  a  wide  range  of 
battlefield  management  applications,  including 
intelligence  production,  situation  assessment,  and 
command  and  control.  It  is  futile  to  attempt  higher  level 
reasoning  for  such  applications  without  possessing  the 
faculty  to  make  snap  judgments  at  the  spatial  perception 
level. 

THREE  NEW  SPATIAL  PERCEPTION  ALGORITHMS 

This  paper  describes  three  new  algorithms 
designed  to  solve  low-leyej  spatial  reasoning  problems 
on  a  map.  Each  algorithm  is  designed  to  emulate  a 
specific  map  reasoning  function  of  the  kind  that  a  human 
can  perform  at  a  glance.  The  first  algorithm,  called 
equidistance  loci-reduced  spatial  discrimination  y  is 
designed  to  associate  an  arbitrary  coordinate  with  the 
nearest  spatial  feature,  under  conditions  when  feature 
boundaries  are  time-varying.  The  second  algorithm, 
called  annulus-based  inclusion  testing  for 
multiply -connected  setSy  is  designed  to  decide  if  an 
arbitrary  coordinate  is  inside  a  closed  contour  (and  if  so, 
where  inside),  when  the  contour  may  be 
multiply-connected.  The  third  algorithm,  called 


topographical  contour  betweenness  testing y  is  designed  to 
label  an  arbitrary  coordinate  with  the  two  topographical 
elevation  contours  between  which  it  is  situated,  and  also 
to  render  the  local  gradient  based  on  interpolation. 

A  tabulation  of  the  techniques  is  presented  below 
at  Table  1.  This  treatise  is  kept  intentionally  at  a 
tutorial  level:  the  mathematical  development  of  the 
algorithms  is  presented  elsewhere  (Ref.  2-4).  It  should 
be  emphasized  that  each  technique  is  deterministic  (i.e., 
computes  a  single,  certain  solution).  The  three  problem 
areas  succumbed  to  deterministic  solutions  because  a 
geometric  approach  produced  compact  data  structures 
which  partition  a  map  based  on  the  spatial  orientation  of 
its  features.  The  discriminating  boundary  of  the 
partition  is  small  in  size  when  compared  to  the  entire 
map,  and  the  resultant  data  compression  guarantees 
feasible  polynomial  complexity;  indeed,  an 
implementation  has  demonstrated  that  each  algorithm 
runs  in  real-time. 


Table  1.  Three  New  Algorithms  for  Command  and 
Control  Reasoning. 


Cartographic  Problem  Area 

Algorithmic  Solution 

Find  nearest  time-varyirg  features 
to  an  arbitrary  Coordinate 

Return  as  a  Vector  the  Distance  and 
Direction  to  the  features 

Equidistanct  Lod-Redvced 

Spatia)  Diicnminatian 

Decide  if  a  coordinate  is  inside  a  closed 
curve  which  contains  other  closed  curves 

Annufus^based  Inclusion  Testing 
for  Multiply-connected  Sets 

Determine  between  Which  TwoEIevation 
Contours  a  Coordinate  Lias 

DecidetheLocatSlopeand  Orientation 
of  the  Coo'dinate 

Topographical  Contour 
“Betweenness  Testing" 

CLOSEST-POINT  PROBLEMS 

Finding  the  closest  point  to  a  given  point  is  a 
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well-studied  problem  in  computer  science.  The 
literature  documents  several  approaches,  including 
Voronoi  diagrams  (Ref.  12,15)  and  chamfering  (Ref.  1,9). 
However,  none  of  the  techniques  directly  addresses  the 
problem  of  locating  the  nearest  time-varying  feature 
boundaries  to  an  arbitrary  coordinate.  The  concept  of 
distance  is  well-developed;  for  an  informative  treatment 
see  reference  10.  The  distance  which  will  be  used  here  is 
variously  known  as  the  city -block  distance,  the 
Manhattan  distance,  or  the  d4  distance  (Ref.  13).  The  d4 
distance  between  two  points  P=(xi,yl)  and  Q  =  (x2,y2)  is 
defined  as  follows: 

d4(P,Q)  =  1X1-X2I  +  Iyi-y2!  (1) 

If  a  Euclidean  metric  is  absolutely  required,  the 
d4  distance  can  first  be  used  as  a  filter  to  find  the  closest 
point  to  a  coordinate;  subsequently  only  the  Euclidean 
distance  to  that  point  alone  need  be  computed.  In  this 
fashion,  relatively  expensive  floating  point  operations  of 
exponentiation  and  multiplication  are  minimized. 

The  Loci-Reduced  Spatial  Discrimination  Algorithm 

Equidistance  loci  reduction  discriminates 
nearness  to  a  feature  by  fencing  in  the  feature  with  the 
equidistance  loci  contour  induced  by  considering  the 
feature  pairwise  with  every  other  feature  on  the  map. 
The  concept  is  informally  described  as  follows.  Select  an 
arbitrary  point  on  the  boundary  of  some  feature. 
Consider  moving  orthogonally  away  from  the  tangent  of 
the  point  with  the  contour  until  the  halfway  point  to  the 
nearest  feature  (or  the  edge  of  the  map)  is  encountered. 
Remember  the  halfway  point.  Continue  the  process  for 
every  point  of  the  feature  boundary.  The  resultant  set  is 
called  the  equidistance  loci  contour  for  the  feature.  Now 
repeat  the  operation  for  every  other  feature  in  the  map 
space.  Unite  the  equidistance  loci  contours  across  all 
features  -  the  union  is  called  the  equidistance  loci  set  for 
the  map  feature  space.  An  example  of  a  loci  set  is  shown 
at  Figure  1.  In  practice,  the  outline  of  a  feature  is  traced 
in  over  a  digitized  map  with  a  man-machine  interface.  A 
smoothing  algorithm  removes  gaps  created  by  a  user's 
hand  moving  faster  than  the  video  sampling  rate.  The 
resultant  linked  list  of  contiguous  coordinates  is  stored 
internally  by  the  computer,  and  is  called  the  trace 
contour  associated  with  the  feature. 


L23  T1  LI  3  LI  2  L23  T2 


Figure  1.  The  loci  set  (in  bold)  displayed  for  a  map  space 
containing  three  features.  The  loci  contour  for  feature 
T3  is  delineated  by  the  line  encompassing  the  arrow  tips. 
Any  coordinate  within  this  contour  is  nearer  feature  T3 


than  any  other  feature. 

The  Computational  Complexity  of  Loci  Reduction 

Theorem.  Equidistance  loci  reduction  is  of  feasible 
polynomial  complexity. 

Proof:  Assume  a  binary  map  space  of  n  features,  each 
represented  by  a  trace  contour  of  contiguous  Cartesian 
coordinates.  From  an  arbitrary  coordinate  (x,y)  the 
algorithm  must  perform  S  1  Ti  I,  i  =  l,  n  distance 
calculations  of  the  form:  I  x  -  xj  I  -H  I  y  ’  yj  I  >  and  also  for 
each  feature  Ti  must  perform  I  Ti  I  comparisons  to  locate 
the  smallest  such  distance.  In  addition,  n  comparisons 
are  required  to  locate  the  smallest  distance  across  n 
features.  Since  the  expressions  do  not  contain  n  as  an 
exponent,  the  algorithm  is  polynomially  complex. 
Furthermore,  the  algorithm  uses  integer-valued 
coordinates;  packs  both  x  and  y  coordinates  into  a  single 
computer  word;  and  avoids  operations  higher  in  the 
computational  hierarchy  than  addition,  subtraction,  and 
absolute  value.  Therefore,  loci  reduction  is  of  feasible 
polynomial  complexity. 

A  Geometric  Nearness  Criterion 

The  equidistance  loci  set  defined  on  a  feature  space 
provides  a  geometric  rationale  for  employing  a 
nearest-neighbor  technique.  Nearness  of  an  arbitrary 
coordinate  to  a  feature  is  defined  naturally  by 
minimizing  the  distance  across  all  trace  contours.  This 
process  is  conceptually  equivalent  to  checking  to  see  if  a 
point  lies  within  the  bounds  of  the  equidistance  loci 
contour  of  a  feature. 

Definition.  A  point  P  is  nearest  to  feature  Fi  with 
associated  trace  contour  Ti  iff 
d4(P,Ti)  <  d4(P,Tj)  V  j  ^  i. 

Definition.  An  object  moves  from  the  vicinity  of  feature 
Fi  to  the  vicinity  of  Fj  iff  the  object  crosses  loci  contour 
Lij. 

Contrasting  Loci  Reduction  with  Voronoi  Diagrams 

The  equidistance  loci  reduction  process  induced  by 
a  feature  boundary  space  is  distinct  from  the  Voronoi 
diagram  induced  by  a  point  set  for  two  related  reasons. 
First,  loci  reduction  operates  on  sets  of  curved, 
contiguous  boundary  point  sets,  whereas  the  traditional 
Voronoi  diagram  constructor  operates  on  random  point 
sets  and  simple  polygons.  Secondly,  loci  reduction  does 
not  look  for  all  points  closer  to  a  given  point  than  any 
other  point;  indeed,  it  may  be  argued  that  the 
contiguous  neighbors  on  a  connected  boundary  meet  this 
criterion.  Rather,  given  an  arbitrary  map  coordinate, 
loci  reduction  searches  every  feature  boundary  for  the 
point  closest  to  the  coordinate,  and  then  selects  the  point 
which  is  closer  than  all  the  others.  The  process  is 
repeated  for  every  point  on  the  feature  boundary.  It  is 
clear  that  the  feature  boundary  space  processed  by 
equidistance  loci  reduction  is  more  general  than  the 
random  point  sets  or  simple  polygons  processed  by 
traditional  Voronoi  diagram  constructors. 
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Contrasting  Loci  Reduction  with  Chamfering 


Chamfering  is  a  technique  which  computes 
nearness  by  exploiting  the  speed  of  prearchived  table 
lookup.  On  a  map  containing  mXn  pixels,  for  each 
feature  an  array  of  mXn  cells  is  precomputed  prior  to  run 
time,  where  each  cell  contains  the  distance  to  the 
feature.  The  technique  is  efficient  for  manageable 
numbers  of  static  features,  since  it  avails  itself  of  table 
lookup,  which  is  a  random  access  operation. 
Regrettably,  in  scaled-up  domains,  .the  number  of  spatial 
features  rapidly  becomes  unwieldy.  For  example,  a  map 
space  of  size  700X700  pixels  with  twenty  binary  features 
uses  twenty  arrays,  each  containing  490,000  bits.  As 
the  number  of  features  increases,  paging  becomes 
inevitable,  which  leads  to  a  thrashing  problem  when 
chamfer  arrays  must  be  continuously  swapped  back  and 
forth  between  physical  and  virtual  space.  Another 
drawback  of  chamfering  is  that  when  a  feature  boundary 
alters  in  shape  or  position,  a  new  corresponding  chamfer 
array  must  be  computed,  which  is  a  time-consuming 
operation  unsuitable  for  many  applications. 


CLOSED  CONTOUR  INCLUSION  PROBLEMS 
The  Odd/Even  Crossings  Technique 


1  j  Ongina)  Contour 
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Figure  2.  A  Graphic  Illustrating  the  Utility  of  the  Inner 
Annulus.  P2  is  nearer  the  inner  annulus  and  therefore 
inside;  conversely,  PI  is  outside. 

The  Formal  Design  Specification  of  the  Annulus 

Inner  annulus  generation  is  algorithmic.  Refer 
to  reference  3  to  review  the  set  of  mathematical 
equations  specifying  the  software  design  of  the 
algorithm.  The  technique  is  described  here  informally. 
Given  a  closed  digital  contour,  the  first  step  is  to  order 
the  pixels  in  a  counterclockwise  direction.  Starting  at  an 
arbitrary  point,  move  along  the  contour  and  assign  to  the 
annulus  the  pixel  4-connected  to  the  left  of  the  current 
pixel.  Continue  until  the  start  point  is  revisited.  The  set 
of  all  points  produced  during  the  traversal  is  called  the 
set  of  first  order  generator  outputs. 


Deciding  whether  a  point  is  inside  a  closed  polygon  is 
another  problem  area  which  has  been  studied,  but  not  to 
the  degree  required  to  treat  the  multiply-connected  sets 
which  frequent  maps  of  real-world  complexity.  There  is 
an  algorithm  available  which  counts  the  number  of  even 
or  odd  crossings  when  a  line  is  drawn  from  the  point 
through  a  polygon;  if  even,  the  point  is  outside  - 
otherwise  it  is  inside  (Ref.  12,15).  The  algorithm  is  an 
elegant  solution  for  simply-connected  sets,  but  fails  for 
multiply-connected  sets.  This  is  a  problem  when 
working  with  maps,  since  maps  are  notoriously 
multiply-connected.  As  an  example  of  the  odd-even 
crossing  technique  failing  in  practice,  consider  the  case 
of  the  Great  Salt  Lake  in  Utah.  Points  interior  to  the 
lake  would  generate  an  even  number  of  Utah  crossings, 
resulting  in  a  decision  that  points  on  the  Great  Salt  Lake 
are  exterior  to  Utah.  Although  this  decision  is  correct 
from  a  topological  stance,  it  is  flawed  pragmatically. 
Another  limitation  of  the  algorithm  is  that  it  relates 
only  whether  a  point  is  inside  or  not  -  it  does  not  provide 
any  information  about  where  inside. 

The  Annulus-based  Inclusion  Testing  Algorithm 

The  inner  annulus  of  a  closed  contour  is 
conceptually  simple  to  describe.  The  annulus  is  a  data 
structure  generated  by  traversing  the  inside  edge  of  a 
closed  clontour  in  a  counterclockwise  fashion,  and 
collecting  the  points  visited  during  the  traversal.  The 
key  idea  is  that  when  one  moves  along  in  a 
counterclockwise  fashion,  the  inside  of  the  contour  is 
always  to  the  left.  Once  the  annulus  is  generated,  it  is  a 
simple  matter  to  compare  the  distance  from  an  arbitrary 
point  to  the  contour  with  that  to  the  annulus;  if  the  latter 
is  smaller,  the  point  is  decided  to  be  inside  (Figure  2).  As 
part  of  an  affirmative  inclusion  decision,  the  technique 
also  provides  both  the  distance  and  direction  to  the 
nearest  point  on  the  inside  edge  of  the  closed  contour. 


There  are  special  cases  which  require  second  order 
treatment,  due  to  peculiar  morphological  structures 
which  give  rise  to  contour  shape.  First,  a  convex  corner 
triplet  is  an  outside  corner  of  the  contour;  i.e.,  it  is  a  set  of 
three  pixels  such  that  the  middle  pixel  forms  a  corner 
with  its  4-connected  neighbors  when  traversed  in  a 
counterclockwise  direction.  Convex  corner  triplets  are 
problematic  because  they  produce  spurious  points 
located  on  the  original  contour.  LetT*  represent  the  set 
of  all  first  points  of  convex  corner  triplets.  Next,  a  local 
concavity  can  be  conceptualized  as  a  "dent"  in  the 
original  contour.  Such  phenomena  require  second  order 
operators  to  add  additional  points  to  the  first  order 
annulus  outputs.  Let  C  denote  the  set  of  points  added. 
Finally,  the  annulus  generator  function  may  produce 
pixels  multiple  times  -  these  points  should  be  entered 
only  once  into  the  annulus  structure.  Let  D(k)  denote 
the  set  of  pixels  produced  at  least  k  times,  where  k  is 
between  2  and  4  inclusive. 

A  Conjecture:  The  Length  of  the  Inner  Annulus 

For  storage  requirement  considerations,  it  is  of 
interest  to  derive  the  length  of  the  annulus  as  a  function 
of  the  length  and  shape  of  the  original  closed  contour. 
Suppose  that  the  original  closed  contour  is  of  length  n 
pixels;  contains  iCI  local  concavities;  IT*I  convex  corner 
triplets;  and  that  the  number  of  pixels  produced  at  least 
k  times  by  the  annulus  generator  function  Ig  is  equal  to 
ID(k)l.  It  is  conjectured  from  empirical  evidence  that  the 
length  of  the  inner  annulus  is  equal  to  the  following 
expression: 

4 

llgl  =  n  +  ICI  -  IT*I  -  S  ID(k)|  (2) 

k  =  2 
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An  example  of  inner  annulus  generation  and  respective 
validation  of  the  conjecture  is  shown  at  figure  3. 

It  is  further  conjectured  that  the  length  of  an 
outer  annulus  Og  (a  structure  for  which  the  contour 
itself  is  the  inner  annulus)  is  an  expression  of  the  same 
form,  with  the  arithmetic  signs  of  the  concavity  and 
convexity  expressions  reversed: 

4 

lOffI  =  n-  ICl  +  1T*1  -  SIDiWl  (3) 

k=2 

Informally,  the  rationale  for  this  conjecture  is  that  what 
is  a  concavity  from  the  inside  of  a  contour  is  a  convexity 
from  the  outside. 
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Figure  3.  An  example  showing  the  generation  of  the 
inner  annulus  for  a  closed  contour  of  length  38.  There 
are  six  convex  corner  triplets,  five  local  concavities,  nine 
points  visited  at  least  twice,  and  one  visited  at  least 
three  times.  Conjecture  2  predicts  the  length  of  the 
annulus  to  be  38  -  6  +  5  -  9  -  1  -  0  =  27,  which  is  in  fact 
the  result  obtained  in  practice. 


The  Topographical  Contour  Betweenness  Algorithm 

Topographical  elevation  contours  are  in  fact 
spatial  features,  similar  to  political  or  geographic 
boundaries.  As  such,  they  can  be  traced  over  a  map,  and 
therefore  will  succumb  to  the  contour  entry  process 
described  above  in  the  section  which  introduces  the  loci 
reduction  process.  The  "betweenness"  algorithm 
actually  makes  two  calls  to  the  loci  reduction  algorithm 
to  return  the  nearest  two  elevation  contours  to  a 
coordinate. 

Regrettably,  nearness  is  not  sufficient  to 
guarantee  betweenness.  However,  the  nearer  of  the  two 
contours  is  guaranteed  to  bound  the  coordinate's 
elevation  from  either  above  or  below  (for  a  proof  see 
reference  4).  Which  way  the  coordinate  is  bound  is  a 
function  of  whether  or  not  the  line  segment  drawn  from 
the  coordinate  to  the  second  contour  crosses  the  first 
(nearer)  contour.  If  the  segment  does  not  cross  the  first 
contour,  then  the  coordinate  is  between  the  first  and  the 
second  contour.  If  the  contours  are  of  equal  elevation, 
then  the  coordinate  is  on  a  saddle  or  in  a  culvert.  If  the 
line  segment  crosses,  and  the  first  contour  is  higher  than 
the  second,  then  the  coordinate  is  between  the  first 
contour  and  one  which  is  k  units  higher  (if  a  higher  one 
does  in  fact  exist  -  otherwise  the  coordinate  is  on  high 
ground);  if  it  crosses  and  the  first  contour  is  lower  than 
the  second,  then  the  coordinate  is  between  the  first 
contour  and  the  one  which  is  k  units  lower  (if  a  lower 
one  does  in  fact  exist  -  otherwise  the  coordinate  is  on  low 
ground).  Figure  4  depicts  the  logic  flow  of  the 
topographical  contour  betweenness  algorithm. 


TOPOGRAPHICAL  BETWEENNESS 


Computing  the  local  orientation  of  an  arbitrary 
point  on  a  topographical  map  is  a  problem  area  which 
has  not  received  the  attention  paid  to  either  closest-point 
or  closed  contour  inclusion  problems.  The  topographical 
orientation  problem  may  be  posed  as  follows:  given  a 
topographical  map  with  an  elevation  contour  resolution 
of  k  units,  return  the  elevations  of  the  two  contours 
between  which  an  arbitrary  point  lies;  also  return  an 
interpolated  local  gradient,  and  the  direction  to  the 
higher  of  the  two  contours. 

Currently  available  algorithms  which  perform 
elevation  reasoning  are  implemented  essentially  in  the 
same  manner  as  chamfering  is  for  closest-point 
problems:  the  elevation  values  at  each  pixel  in  a  digital 
map  are  prearchived;  table  lookup  is  used  at  run-time  to 
render  the  elevation  at  a  particular  point.  There  is  a 
need  for  higher  order  logic  to  reason  across  elevation 
contours. 


Figure  4.  A  Flowchart  of  the  Topographical  Contour 
Betweenness  Algorithm. 

INTEGRATING  THE  ALGORITHMS 


Map  reasoning  requires  an  understanding  of 
feature  orientation  and  nearness,  an  appreciation  of  how 
one  spatial  feature  can  be  part  of  another,  and  a 
capability  to  decipher  a  maze  of  topographical  contours 
into  the  corresponding  three-dimensional  reality. 
Equidistance  loci  reduction  is  a  geometric  interpretation 
of  feature  orientation  and  nearness.  Annulus-based 
inclusion  testing  for  multiply-connected  sets  is  a 
topological  tool  which  treats  the  set  inclusion  problem. 
Betweenness  testing  is  a  geometric  tool  which  addresses 
the  point-to-point  directional  gradient  problem. 

When  packaged  together,  the  three  algorithms 
comprise  a  set  of  tools  which  can  deterministically 
answer  many  of  the  low-level  spatial  reasoning 
questions  which  are  taken  for  granted  by  human 
analysts.  For  a  command  and  control  operation,  which 
has  a  requirement  to  know  how  both  red  and  blue  forces 
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are  oriented  upon  a  map  of  spatial  features,  the  toolkit 
provides  a  real-time  capability  to  put  a  universal 
transverse  Mercator  projection  (UTM)  coordinate  into 
context,  based  on  the  distance  and  direction  to  objects  in 
the  vicinity.  Because  the  algorithms  are  of  generic 
mathematical  design,  they  are  immediately 
leverageable  against  any  digitized  map  of  interest. 

IMPLEMENTING  THE  PACKAGE 


The  three  algorithms  have  been  implemented 
within  an  Interlisp  environment,  and  have  been  applied 
to  two  styles  of  binary  maps.  The  first  set  of  maps,  used 
to  demonstrate  loci  reduction  and  annulus-based 
inclusion  testing,  is  from  the  World  Data  Bank  II 
(WDBII),  which  contains  features  consisting  of  political 
borders,  coastlines,  rivers,  lakes,  and  mountain  chains. 
The  second  set  of  maps,  used  to  demonstrate  the 
topographical  contour  betweenness  algorithm,  is  from 
the  United  States  Geographical  Service  (USGS) 
archives,  and  in  addition  to  portraying  a  higher 
resolution  version  of  the  same  features  contained  in 
WDBn,  the  set  includes  buildings,  roads,  and  20  foot 
elevation  contours. 

The  maps  employed  are  of  real  world  complexity; 

i.e.,  a  single  map  displayed  on  half  a  CRT  screen 
consumes  500,000  bits  of  information,  and  depending 
upon  the  local  geographic  and  political  profile,  the 
number  of  features  can  be  large.  Although  only  three 
WDBII  maps  and  one  USGS  map  have  been  analyzed, 
the  performance  of  the  algorithms  is  timely.  The 
response  to  a  typical  query  requires  five  milliseconds  of 
CPU  time.  The  performance  potential  of  the  map 
reasoning  package  has  attracted  the  attention  of 
operations  personnel,  and  as  a  result  the  developmental 
software  for  loci  reduction  and  annulus-based  inclusion 
testing  has  recently  been  ported  to  advanced 
development. 

CONCLUSIONS 

Three  new  algorithms  have  been  developed  to 
emulate  specific  functions  of  the  spatial  perception 
process  experienced  by  humans  when  interpreting  a  map 
during  command  and  control  processing.  The  first 
algorithm,  called  equidistance  loci  reduction^  partitions  a 
map  with  a  discriminating  boundary  consisting  of  the 
equidistance  locus  formed  by  considering  each  feaure 
pairwise  with  every  other  feature  on  the  map.  This 
technique  is  designed  to  associate  an  arbitrary 
coordinate  with  the  nearest  spatial  feature,  even  as 
feature  boundaries  vary  in  time.  The  second  algorithm, 
called  annulus-based  inclusion  testing  for 
multiply -connected  sets,  provides  a  technique  to  decide  if 
a  point  is  inside  a  closed  contour,  when  the  closed 
contour  itself  contains  other  closed  contours.  The 
technique  is  important  to  command  and  control  because 
maps  are  multiply-connected.  The  third  algorithm, 
called  topographical  contour  betweenness  testing,  is 
designed  to  decide  between  which  two  elevation  contours 
a  point  sits,  and  as  a  corollary  computes  the  local 
directional  gradient.  Taken  as  a  package,  the  set  of 
algorithms  deterministically  answers  many  of  the 


perceptual  level  questions  which  a  command  and  control 
system  requires  to  progress  with  higher  level  reasoning. 
Since  the  algorithms  are  of  feasible  polynomial 
complexity,  the  compiled  implementation  returns 
spatial  snap  judgments  in  real-time,  which  is  a 
requirement  for  front-end  perception  during  command 
and  control  processing. 
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ABSTRACT 


A  new  approach  to  resolving  ambiguities  of  a  numerical  ttacker  provides  eiAanc^  tracbng 
operator  workload'.  This  approach  utilizes  combined  symbolic  and  numerical  algorithms  w 
sy^stem  based  on  Dempster-Shafer  theory.  Simulation  results  demonstrate  successful  performance 


capability  and  yields  reduced 
interact  with  an  identification 
of  the  system. 
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INTRODUCTION 


This  abridged  paper  illustrates  the  performance  gains 
achievable  by  integrating  expert  systems  and  numerical  track¬ 
ing  algorithms.  The  resultant  tracking  expert  system  (ES) 
reasons  with  heuristic  rules  based  on  target  identification 
(ID),  threat  level,  collateral  information,  kinematic  ID  dis¬ 
criminants,  and  operator’s  knowledge. 

Target  ID  information  is  derived  from  sensor  reports  as 
well  as  from  collateral  information,  operator  heuristics,  and 
target  group  (kinematic)  behavior.  This  aggregate  data  is 
processed  by  a  separate  ID-ES  based  on  Dempster/Shafer 
(D/S)  theory. 

In  addition,  this  paper  illustrates  that  the  combined  system 
can  reduce  spurious  tracks  not  resolvable  by  the  numerical 
tracker  alone.  It  should  be  noted  that  this  system  concept  is  a 
forerunner  to  an  “intelligent  tracker”  which  will  utilize  tem¬ 
poral  and  spatial  reasoning  along  with  contextual  knowledge 
to  resolve  track  ambiguities. ' 

BACKGROUND 

Most  numerical  tracking  algorithms  (Ref.  1)  are  unable  to 
resolve  closely  spaced  target  tracks,  such  as  crossing  targets 
in  unfavorable  sensor  coordinates  or  during  evasive  target 
maneuvers.  In  these  circumstances,  one  target  may  be  associ¬ 
ated  with  multiple  tracks,  one  track  may  be  associated  with 
multiple  targets,  or  a  target  may  be  incorrectly  assigned  to  a 
track. 

Yannone  (Ref.  2)  discusses  recent  trends  in  multisensor/ 
multitarget  tracking  using  probabilistic  association  combined 
with  artificial  intelligence  (AI)  techniques.  Bonissone  (Ref. 
3)  developed  a  tracking  ES  to  resolve  track  ambiguities  but 
did  not  use  ID  information.  Kadar  (Ref.  4)  developed  an  iter¬ 
ative  evidential  reasoning  scheme  based  on  a  perceptual  rea¬ 
soning  paradigm  to  enhance  the  performance  of  both 
Situation  Threat  Assessment  Response  Strategy  Systems 
(STARS)  and  the  ID  and  tracking  subsystems  thereof. 

This  paper  generalizes  previous  approaches  by  the  intro¬ 
duction  of  an  interactive  tracking  ES  and  ID-ES. 


APPROACH 


The  decision  aid  scheme  introduced  in  this  paper  is  differ¬ 
ent  from  previous  approaches.  It  consists  of  interactive  track¬ 
ing  ES  and  ID-ES  that  reduce  both  track  and  ID  ambiguities 
by  the  combination  of  numerical  and  symbolic  algorithms.  By 
using  kinematic  ID  discriminant  data  to  amend  the  input  vec¬ 
tor  of  the  ID  system  (concatenating  the  ID  sensor  data  input 
vector  with  the  kinematic  discriminants),  the  performance  of 
the  iterative  system  (Ref.  4)  can  be  enhanced  with  reduced 
iteration  steps.  Track  kinematic  ID  discriminants  are  derived 
from  expected  measurement  parameters  such  as  target  speed 
and  altitude  regions,  maneuvers,  and  target  location  with  re¬ 
spect  to  an  apriori-known  flight  corridor,  etc.  Thus  the  inputs 
to  the  ID-ES  consist  of  probabilistic  declarations  based  on 
comparing  the  extracted  sensor  data  feature  parameters  with 
an  apriori  correlation  data  base. 

The  rule-based  tracking  ES  is  developed  to  assist  numerical 
algorithms  (branch  and  bound  and  nearest  neighbor),  imple¬ 
mented  at  the  Grumman  Artificial  Intelligence  Lab,  to  reduce 
track  ambiguities.  These  heuristic  rules  are  based  on 
experienced-pilot  knowledge,  target  ID  and  threat  priority  in¬ 
formation,  common  sense,  and  collateral  information.  The 
tracking  ES  interacts  with  the  numerical  tracking  routines  and 
the  identification  ES.  Figure  1  shows  the  conceptual  system 
diagram. 


Fig.  1  Tracking  Expert  System 
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The  outputs  of  the  numerical  tracking  routines  are  simu¬ 
lated  using  the  Automated  Reasoning  Tool  (ART),  which  is 
an  expert  system  development  tool.  The  temporal  reasoning 
capability  of  ART  allows  processing  of  the  time-varying  track 
states.  The  rules  based  on  spatial  reasoning  are  used  to  iden¬ 
tify  tracks  that  could  merge  and  times  when  the  tracks  would 
merge  and/or  split.  The  heuristic  rules  resolve  the  track  am¬ 
biguities,  and  the  results  are  displayed  for  an  operator.  The 
rules  based  on  target  ID  are  assigned  the  highest  priority. 

The  ID-ES  block  diagram  is  depicted  in  Fig.  2.  This  sys¬ 
tem  provides  ID  of  targets  by  combining  information  from 
multiple  sensors.  The  target-feature-discriminant  parameters 
are  extracted  from  individual  sensor  declarations,  including 
the  kinematic  features  described  before,  and  are  compared 
with  a  sensor/target  feature  parameter  correlation  data  base 
(intelligence  files).  The  correlation  process  is  uncertain  due 
to  both  measurement  noise  and  the  non-uniqueness  property 
of  the  correlation  data  base. 


Fig.  2  Target  Identification  Expert  System 


The  system  simulates  extracted  target-feature  parameters 
that  can  be  derived  from  sensor  measurements/reports.  The 
system  assigns  a  distribution  functional  to  each  target-feature 
parameter.  Targets  to  be  identified  have  their  respective 
feature-discriminant  parameters  stored  in  the  target  data  base. 

The  uncertainties  among  the  target(s)  and  associated  fea- 
ture(s)  are  computed  based  on  sensor  uncertainties  and  simu¬ 
lated  feature  values. 

The  D/S  evidential  reasoning  scheme  is  used  to  combine 
the  uncertain  evidencies  based  on  emulated  sensor  reports. 
This  data  is  used  in  the  correlation  process  to  yield  a  set  of 
possible  beliefs  assigned  to  targets. 

Both  iterative  (Ref.  4)  and  noniterative  evidence  combina¬ 
tion  schemes  are  implemented.  ART  is  used  in  this  case  to 
implement  heuristic  ID  rules  and  to  store  the  target  knowl¬ 
edge  base  (KB).  The  D/S  algorithm  is  implemented  in  LISP. 
The  kinematic  feature-discriminant  data  from  the  tracking  ES 
is  used  to  concatenate  the  feature  vector  data  based  on  ID 
declarations  in  the  absence  of  track  data,  as  described  previ¬ 
ously.  Thus  the  ID-ES  and  the  tracking  ES  interact  in  a  man¬ 
ner  to  reduce  ambiguities  both  in  track  and  ID, 


PERFORMANCE  EVALUATION 

Figure  3  displays  simulated  outputs  from  branch  and  bound 
and  the  nearest  neighbor  tracking  algorithms.  The  nearest 
neighbor  tracks  continue  in  a  straight  line  after  tracks  cross 
(track  3  and  track  4).  However,  tracks  generated  from  the 
branch  and  bound  routine  change  direction  after  they  cross 
(track  1  and  track  2). 


Fig.  3  Crossing  Tracks  before  Expert 
System  Is  Run 

Figure  4  displays  tracks  after  the  tracking  expert  was  run. 
In  this  case,  track  ambiguities  based  on  target  ID  were  re¬ 
solved.  Track  1  and  track  2  actually  do  not  change  direction, 
but  fly  in  a  straight  line  after  they  crossed,  based  on  their  ID. 
This  information  is  passed  back  to  the  numerical  tracker  and 
ID  expert  system. 


Fig.  4  Crossing  Tracks  after  Expert 
System  Is  Run 


The  ID-ES  output  is  shown  in  Fig.  5.  This  result  is  based 
on  simulated  inputs  from  six  probabilistic  declarations  de¬ 
rived  from  a  sensor  suite  (both  active  and  passive  sensors) 
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Fig.  5  Evidence  Tree 


and  uses  kinematic-discriminant-feature  data.  The  kinematic 
features  included  maximal  altitude  and  speed  of  targets. 

The  system  output,  shown  in  Fig.  5,  is  in  the  form  of  a 
hierarchical  tree  with  beliefs  computed  for  each  node  repre¬ 
senting  one  or  more  targets  or  target  classes.  The  tree  struc¬ 
ture  represents  the  frame  of  discernment  based  on  prior 
knowledge.  The  use  of  prior  knowledge  allowed  for  a  signifi¬ 
cant  reduction  of  the  power  set  of  possible  target  combina¬ 
tions  to  a  denumerable  linear  subset.  In  this  example,  the 
target  identified  is  an  SR71  with  a  belief  of  0.9.  This  result  is 
then  used  in  the  tracking  ES  to  support  the  multitarget  multi¬ 
sensor  track  association  process  and  operator  decision  aiding. 
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ABSTRACT 

This  paper  investigates  methods  tor  enhancing  acquisition  performance  for  an  Atmospheric  Defense  Initiative  (ADI)  ®"8agement  ^tfotro  utiUring  a 
multisensor  system.  A  multisensor  constant  false  alarm  rate  (MSCFAR)  processor  is  described  which  maximizes  overall  engagement  platform  detection  prob¬ 
ability  for  thrmultisensor  suite  by  adaptively  controlling  each  sensor’s  threshold  setting  using  local  background  and  desired  signal  estimates  while  keeping  a 
fUed^alse  alarm  rate  for  the  entire  multisensor  system.  A  computer  simulation  is  used  to  provide  quantitative  results  of  the  benefits  of  multtsensor  adaptive 
control  for  a  system  consisting  of  a  radar  and  a  dual  band  Infrared  Search  and  Track  (IRST)  system. 

The  assumptions  for  the  analysis  are  that  the  radar  target  has  Swerling  III  fluctuation  and  is  imbedded  in  Rayleigh  noise.  The  IR  signature  of  the  target  is 
non^ctuaTngS  fe  Embedded  I  Gaussian  noise.  AR  spatial  and  temporal  alignment  for  each  sensor’s  data  is  assumed  to  have  been  already  accomplished. 

The  quantitative  results  obtained  indicate  that  maximum  probability  of  detection  (Pj)  enhancement  for  a  multisensor  system  is  achieved  when  all  sensors 
are  operating  with  similar  detection  capability.  When  one  sensor  has  a  much  higher  probability  of  detecting  the  target  than  any  of  the  other  sensors,  the  Pj 
enhancement  using  a  multisensor  CFAR  is  not  significant  in  itself:  however,  the  information  obtained  can  be  useful  for  optimally  managing  the  sensors  via 
cueing  (i.e.,  dwell  time  and  field  of  view  management)  to  maximize  overall  mission  success. 
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1.0  INTRODUCTION 

Avionics  performance  requirements  for  next  generation  ADI  weapon 
systems  cannot  be  met  by  conventional  stand-alone  sensor  systems.  The 
projected  operational  requirements  include  long  range  target  detection, 
precision  multiple  target  tracking,  target  classification  and  identification, 
and  a  high  level  of  immunity  from  countermeasures.  No  sin^e  sensor 
system  will  be  able  to  provide  all  of  these  capabilities  against  all  types  of 
targets  under  all  operating  conditions. 

A  significant  potential  performance  improvement  is  available  by  util¬ 
izing  complementary  sensors  such  as  radar,  infrared,  and  electronic  sup¬ 
port  measures  in  an  integrated  manner.  To  achieve  the  highest  level  of 
performance  from  a  multiple  sensor  system,  it  is  necessary  to  do  more 
than  merely  fuse  the  outputs  of  the  individual  sensor  track  file  outputs.  A 
multisensor  system  which  fully  integrates  the  sensor  functions  at  the  signal 
and  data  processing  levels  will  provide  performance  and  cost  benefits  un¬ 
attainable  by  any  other  means. 

The  Atmospheric  Defense  Initiative  is  concerned  with  CONUS  de¬ 
fense  against  incoming  cruise  missiles.  Due  to  the  low  observable  nature 
of  the  threat  and  its  ability  to  fly  at  altitudes  which  create  severe  clutter 


problems  for  radar  and  IR  sensors,  all  sensor  systems  onboard  the  aircraft 
must  be  utilized  in  a  near-optimal  manner  to  effectively  counter  the 
threat. 

The  ADI  mission  provides  a  stressing  scenario  in  which  to  test  some 
new  concepts  being  developed  by  GE  for  the  application  of  multisensor 
systems.  The  baseline  scenario  which  will  be  referred  to  throughout  the 
paper  is  the  engagement  problem  consisting  of  the  surveillance  platform 
initially  detecting  the  target  and  then  handing  off  the  target  to  an  engage¬ 
ment  platform  which  is  responsible  for  acquiring  the  target  and  counter¬ 
ing  it. 

The  focus  of  this  paper  will  be  on  the  development  of  adaptive  con¬ 
trol  over  all  available  sensors,  and  results  will  be  presented  on  the  appli¬ 
cation  of  a  multisensor  constant  false  alarm  rate  processor  for  the  engage¬ 
ment  platform  which  will  utilize  all  information  previously  obtained  by  the 
surveillance  platform  along  with  its  own  data  to  allow  optimal  target  ac¬ 
quisition.  The  multisensor  fire-control  architecture  used  throughout  the 
paper  is  depicted  in  Figure  1  which  indicates  the  various  sensors  which 
are  utilized  along  with  the  assumed  interaction  among  subfiinctions. 


FIGURE  1.  FIRE-CONTROL  MVLTISENSOR  ARCHITECTURE 
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It  is  assumed  that  the  surveillance  platform  provides  the  engagement 
platform  with  information  about  the  target  state  vector  and  covariance 
matrix  which  the  engagement  platform  will  use  to  determine  the  required 
field  of  regard  that  it  must  search  out  to  acquire  the  target.  The  surveil¬ 
lance  platform  also  supplies  information  which  it  has  gathered  about  the 
target  signatures  and  environmental  conditions  for  all  pertinent  spectral 
bands. 

2.0  MULTISENSOR  ADAPTIVE  CONTROL 

The  term  multisensor  adaptive  control  is  defined  as  the  controlling 
function  which  uses  information  from  other  sensors  to  control  some  pa¬ 
rameter  or  parameters  of  another  sensor. 

A  block  diagram  showing  inputs  and  outputs  for  the  function  defined 
as  multisensor  adaptive  control  is  shown  in  Figure  2.  The  mission  inputs 
which  are  typically  supplied  by  the  overall  mission  computer  determines 
the  priority  tasking  (i.e..  which  targets  are  most  threatening  versus  those 
targets  which  are  least  threatening)  and  also  defines  how  many  false 
alarms  the  overall  data  processing  system  can  handle.  The  static  inputs 
consist  of  sensor  parameters  which  do  not  change  quickly  wth  time  such 
as  radar  power-aperture,  frequency  bands,  and  losses.  Static  variables  for 
an  IRST  would  typically  consist  of  sensor  resolution  along  with  the  sensor 
sensitivity  in  search  and  track  modes  assuming  an  uncluttered  blue  sky 
background. 

The  dynamic  variables  which  may  be  chan^ng  as  a  function  of  time 
are  the  number  of  targets  to  engage,  the  sensor  data  inputs,  environ¬ 
mental  conditions,  signature  information  on  the  targets  of  interest,  along 
with  target  dynamics. 

The  multisensor  adaptive  control  function  would  process  signal  and 
background  information  for  each  sensor  cell  for  which  it  has  information 
and  estimate  each  cell’s  signal  to  interference  (SIR)  level  to  optimize 
sensor  suite  performance  for  a  variety  of  different  modes  such  as  surveil¬ 
lance,  track,  or  identification  capability  using  the  available  information 
from  the  onboard  sensors,  along  with  the  constraints  imposed  by  the  mis¬ 
sion  inputs  and  data  processing  system  capabilities. 

The  outputs  which  would  be  fed  back  to  the  individual  sensors  consist 
of  threshold  multipliers  (i.e.,  the  constants  which  multiply  the  individual 
sensor’s  background  estimates),  coverage  information,  timeline  manage¬ 
ment  information,  and  other  information  concerned  with  optimal  sensor 
management. 

Two  examples  to  illustrate  the  principle  of  a  multisensor  CFAR  are 
presented  next.  The  first  example  considers  two  IR  sensors  with  perfect 
cell  alignment  and  similar  statistical  behavior.  In  the  second  example,  a 
radar  with  different  cell  sizes  and  Swerling  III  target  fluctuation  is  added 
to  the  two  IR  sensors  of  the  first  example  to  show  the  effect  of  combining 
information  from  dissimilar  sensors. 

2.1  Dual  Band  IR  System  (Two  Sensor  Example) 

The  first  example  in  optimizing  multisensor  performance  in  the  ac¬ 
quisition  mode  will  be  for  two  IR  sensors  operating  in  Gaussian  noise  with 
a  non-fluctuating  target.  The  target  signature’s  sensor  sensitivities  are  as¬ 
sumed  to  be  different  in  both  bands  of  interest. 

The  assumed  statistical  nature  of  the  two  IR  sensors  spatial  filter 
outputs  is  shown  below  in  Figure  3. 
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FIGURE  3.  SENSORS  WITH  GAUSSIAN  NOISE 
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It  is  assumed  that  both  IR  sensors  are  viewing  the  same  angular  field 
of  view  and  hence  are  perfectly  correlated  in  the  spatial  domain  and  that 
the  data  is  taken  at  close  enough  instants  in  time  so  that  the  target  has 
not  moved  from  one  cell  to  the  next. 

The  current  method  of  detection  processing  for  two  IR  sensors  is 
shown  in  Figure  4.  Current  dual  band  IR  sensor  systems  operate  autono¬ 
mously  with  no  adaptive  or  intelligent  system  optimizing  performance  for 
the  overall  mission. 

The  following  is  an  investigation  of  the  treatment  of  the  multisensor 
suite  as  a  single  system  with  the  constraint  of  a  constant  false  alarrn  rate 
for  the  entire  group  of  sensors,  and  the  optimality  criteria  of  maximum 
overall  detection  probability  is  the  quantity  which  will  be  maximized. 

The  overall  architecture  for  the  multisensor  CFAR  is  shown  in  Figure 
5.  The  MSCFAR  uses  estimates  of  the  signal  to  interference  ratio  for 
each  cell  and  then  determines  the  optimal  method  of  selecting  each  indi¬ 
vidual  sensor  threshold  so  that  a  constant  overall  system  probability  of 
false  alarm  (Pf^)  can  be  maintained. 

The  motivation  behind  the  MSCFAR  is  that  to  the  author’s  knowl¬ 
edge,  no  multisensor  system  currently  in  operation  combines  information 
from  all  sensors  to  optimize  performance.  A  practical  example  of 
MSCFAR  operation  is  the  case  of  two  sensors  arbitrarily  set  to  the  same 
Pfg  assuming  equal  performance.  In  reality,  the  signature  of  the  target  is 
much  higher  for  one  sensor  than  for  the  other  (i.e.,  a  hot  IR  target  will 
have  a  stronger  signature  in  the  IR  band  at  the  shorter  wavelength); 
therefore,  the  MSCFAR  would  raise  the  Pf^  (lower  the  threshold)  for  the 
higher  signature  sensor  (maximizing  P^j)  and  lower  the  Pf^  (raising  the 
threshold)  for  the  other  sensor  to  maintain  constant  system  Pf^ . 


DYNAMIC  INPUTS: 


.  SENSOR  DATA 
Xr  (R.Pd.Az.EI)  Xir  (Az.El) 
•  ENVIRONMENTAL  INFO 
»  SIGNATURE  INFO 


STATIC  INPUTS: 

►  RADAR  PARAMETERS  (P.A.f.L) 
t  IRST  PARAMETERS  NEI  (8KGD, 
FOR,  FRAME  TIME) 


•  ESTIMATE: 

-  (S/l)  rad.  (S/l)  ir 

-  Irad  (R,  Fd.  Az.  El) 

-  lir  (Az.  El) 

•  OPTIMIZE; 


-  TRACK  ACCURACY 

-  IDENTIFICATION 
►  CONSTRAINTS: 

CONSTANT  P,g 


OUTPUTS; 


I  SENSOR  THRESHOLDS 
»  SENSOR  COVERAGE 
I  SENSOR  TIME-LINE  MANAGEMENT 

-  DWELL  TIMES 

-  FRAME  TIMES 

•  RADAR  WAVEFORM  SELECTION 

•  SENSOR  SIGNAL  PROCESSING 
»  SENSOR  CUEING 

•  CORRELATION  CRITERIA _ 


FIGURE  2.  MULTISENSOR  ADAPTIVE  CONTROL  FUNCTION 
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Ki  =  THRESHOLD  MULTIPLIER  FOR  BAND  1 


THRESHOLD 

CROSSING 

REPORT  FOR  BAND  1 


K2  =  THRESHOLD  MULTIPLIER  FOR  BAND  2 


FIGURE  4.  CURRENT  DUAL  BAND  IR  DETECTION  PROCESSING 


FIGURE  5.  MULTISENSOR  CFAR  CONCEPT  FOR  DUAL  BAND  IRST 
■  1  out  of  2  (assuming  Gaussian  noise) 


•  Maximize:  Pd  =  1  -  Pm1  •  Pm2  =  1  -  / 


KfOi 


•  Constraint:  Pfa  =  1  ~  [(1  -  Pfal)(1  -  Pfa2)I 

CO  -x2 

“Pfa1  +  Pfa2=  /  ^  ^20^2  + 


20^2 
_e _ 


ioo  e 

7^02  ^ 


Ki*  01 


-Y2 
2  022 


dx  K2*02 


■  2  out  of  2 

•  Pd  =  Pd1  •  Pd2 

•  Pfa  =  Pfal  •  Pfa2 


•  Can  Lower  Thresholds  Considerably  When  Requiring  Two 
Sensor  Correlation 


FIGURE  6.  TWO  SENSOR  DETECTION  WITH  CORRELATION 


For  the  cases  where  there  is  a  one-to-one  cell  mapping,  as  for  per- 
fecdy  aligned  IR  sensors  with  the  same  instantaneous  fields  of  view,  the 
optimal  thresholds  can  be  calculated  in  two  different  ways.  One  sensor 
could  be  required  to  have  a  threshold  crossing  (i.e.,  the  1  of  2  criteria), 
or  both  sensors  could  be  required  to  have  a  threshold  crossing  for  a  de¬ 
tection  to  take  place  (i.e.,  the  2  of  2  criteria).  It  is  intuitively  obvious  that 
the  thresholds  can  be  set  much  lower  when  requiring  the  2  of  2  criteria 
while  still  maintaining  the  same  overall  system  Pfg . 

The  equations  relating  the  probabilities  of  detection  and  the  prob¬ 
abilities  of  false  alarm  are  shown  in  Figure  6. 


To  solve  for  the  optimal  threshold  multipliers  K1  and  K2,  a  computer 
simulation  was  developed  which  searches  over  the  constant  manifold 
given  by  the  preceding  equations  for  the  threshold  multipliers  which  yield 
the  highest  system  P^  for  the  fixed  false  alarm  probability  specified  for 
the  overall  sensor  suite. 

To  get  a  feel  for  what  the  results  should  look  like  for  the  1  of  2 
criteria,  the  overall  system  Pj^  was  set  to  0.0001  and  the  estimated  SIRs 
are  assumed  to  be  SIR  1=4.0  and  SIR2  =  5.0. 

Then,  as  Pf^j  is  varied,  can  be  solved  for  directly  as: 
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SIR2  =  5  TOTAL  Pfg  =10 


FIGURE  7.  TWO  SENSOR  EXAMPLE 


Pfa2  =  0.0001  -  Pfai 

The  graph  of  overall  P^j  is  shown  in  Figure  7. 

V’isually  one  can  pick  the  individual  Pf^s  where  maximum  P^  occurs 
to  be  Pfai  =  0.00003  and  Pfa2  =  0.00007.  This  makes  intuitive  sense 
(i.e.,  a  sensor  with  higher  SIR  should  be  allowed  more  chance  of  giving  a 
false  alarm  since  more  information  is  being  obtained  from  that  sensor). 
For  this  example,  it  is  seen  that  only  4  percent  is  gained  by  choosing  the 
optimal  set  of  thresholds  using  the  1  of  2  criteria  rather  than  just  allowing 
the  dominant  sensor  (the  one  with  higher  estimated  SIR)  to  have  all  of 

the  Pf a . 

Including  the  option  of  a  2  of  2  criteria  selection  requires  that  the 
optimal  be  calculated  for  both  criteria  and  then  the  criterion  resulting 
in  the  highest  P^  be  selected  for  use. 

A  representative  example  using  both  criteria  is  shown  below  in  Table 
1.  This  example  shows  that  improvements  in  detectivity  on  the  order  of 
13  percent  can  be  achieved  when  sensors  are  working  at  similar  detection 
performances  using  the  2  of  2  criteria. 

By  searching  over  the  constant  Pfa  manifolds  for  both  criteria  as  a 
function  of  input  SIRs,  the  selection  region  for  a  system  requiring  an 
overall  Pfa  of  0.00001  is  shown  in  Figure  8,  along  with  the  overall  maxi¬ 
mum  which  was  achieved. 

The  following  cases  shown  in  Figures  9  and  10  show  that  the  criteria 
selection  region  changes  as  the  requirement  of  overall  Pfa  changes  and 
the  2  of  2  criteria  pro^des  better  overall  system  P^  along  the  region  when 
both  sensors  are  operating  at  relatively  similar  SIRs.  The  basic  trend 
which  is  shown  when  the  overall  system  Pfg  is  made  smaller  is  that  the  2 
of  2  criteria  is  generally  more  often  used. 


2.2  Dual  Band  IRST  and  a  Radar  (3  Sensor  Example) 

The  next  example  uses  the  two  IR  sensors  from  the  last  example  and 
a  radar  which  has  different  statistical  properties  than  the  IR  sensors  and 
whose  cell  dimensions  do  not  match  perfectly  with  the  two  IR  sensors. 

The  statistical  properities  of  the  radar  are  assumed  to  be  Rayleigh 
noise  along  with  a  Swerling  III  fluctuating  target. 

Since  the  radar  cell  may  be  larger  in  the  angular  dimension  than  that 
of  the  IR  sensors  along  with  the  additional  dimensions  of  range  and  Dop¬ 
pler  as  shown  in  Figure  11,  perfect  correlation  from  a  detection  cell  in 
radar  to  one  in  an  IR  sensor  does  not  occur.  However,  one  can  still  ask 
the  question,  “How  should  the  MSCFAR  choose  the  optimal  thresholds 
when  a  threshold  crossing  is  required  in  at  least  one  of  the  three  sensors 
in  order  to  declare  that  a  detection  occurred?” 

Figure  12  indicates  how  the  individual  sensor  false  alarm  probabilities 
and  single  hit  detection  probabilities  vary  as  a  function  of  range  for  a 
representative  set  of  sensor  sensitivities  and  target  signature  values.  As 
can  be  seen  from  Figure  12,  the  radar  is  the  dominant  sensor  due  to  its 
fluctuating  target  at  long  range,  but  as  range  decreases,  the  IR  sensor 
operating  in  band  2  begins  to  detect  the  target  and,  since  its  target  is 
non-fluctuating,  its  P^f  curve  has  a  much  faster  slope  (even  though  the 

SNR  varies  as  1/R^  versus  1/R^  for  the  radar),  eventually  overtaking  the 
radar  as  the  dominant  sensor.  The  overall  P^j  is  seen  to  be  significantly 
higher  than  each  of  the  individual  sensor’s  P^js  for  cases  where  the  indi¬ 
vidual  sensors  have  approximately  the  same  P^;  however,  the  overall  P^j 
generally  tracks  the  dominant  sensor  when  one  sensor  provides  a  much 
higher  P^^  than  the  other  two.  The  latter  result  is  shown  better  in  Figure- 
13  where  the  radar  still  has  higher  P^  for  longer  ranges  but  the  IR  sensor 
in  band  2  begins  to  detect  the  target  much  earlier  than  before  and  gener¬ 
ally  is  the  dominant  sensor  over  most  of  the  useful  range  of  the  sensor 


TABLE  1.  TWO  SENSOR  EXAMPLE 


1  out  of  2  Correlation  Criteria: 

Thi  =  4.0  Th2  =  3.8 


=  0.0001 


PdT, 


Pdi  =  0.49 


Pd2  ~  0.88 


2  out  of  2  Correlation  Criteria: 

Thi  =  2.0  Th2  =  2.6 


Pdi  =  0.97 


Pd2  =  0.99 


PdT 


■  Example  2:  SNR  1  =  3  SNR  2  =  3 
•  1  out  of  2  Correlation  Criteria: 

Thi  =  3.9  Th2  =  3.9 


PfaT 


PdT 


Pdi  =  0.19 


Pd2  =  0,19 


•  2  out  of  2  Correlation  Criteria; 

Thi  =  2.3  Th2  =  2.3 


Pdl  =  0.75 


Pd2  =  0.75 


PdT 


APd  =0.02 


APd  =  0.13 
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SELECTION  REGION 


COMBINED  P, 


Pd  SCALE 


FIGURE  8.  TWO  SENSOR  MAXIMUM  PROBABILITY  OF  DETECTION 
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PROBABILITY  OF  FALSE  ALARM  =  0.0001 
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2  OUT  OF  2  DETECTION  CRITERIA 


1  OUT  OF  2  DETECTION  CRITERIA 


FIGURE  9.  TWO  SENSOR  CORRELATION  SCHEME  FOR 
MAXIMUM  PROBABILITY  OF  DETECTION  CRITERIA 


■  One-to-one  Cell  Mapping 

•  IR  Sensors  with  identical  Instantaneous  fields-of-view 


■  Many-to-many  Cell  Mapping 
•  Many  IR  Celis  corresponding  to  many  Radar  Cells 


ANGLE 

ONLY 


ANGLE 

RANGE 

DOPPLER 


FIGURE  II.  SENSOR-TO-SENSOR  CORRELATION 


2  OUT  OF  2  DETECTION  CRITERIA 


1  OUT  OF  2  DETECTION  CRITERIA 


FIGURE  10.  CORRELATION  SCHEME  FOR  MAXIMUM 
PROBABILITY  OF  DETECTION  CRITERIA 


SINGLE  HIT  DETECTION  PERFORMANCE 


FIGURE  12.  MULTISENSOR  CFAR  EXAMPLE  I 
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SINGLE  HIT  DETECTION  PERFORMANCE 


ADAPTIVE  FALSE  ALARM  PROBABILITIES 


figure  13.  MULTISENSOR  CFAR  EXAMPLE  2 


suite.  The  enhancement  in  Figure  13  is  less  than  that  shown  in  Figure 
12  because  over  most  of  the  range,  one  sensor  dominated  in  terms  of 
overall  detection  capability. 

Although  the  P^j  enhancement  is  not  significant  in  all  cases,  the  infor¬ 
mation  which  is  obtained  from  the  MSCFAR  can  be  used  very  effectively 
for  optimal  sensor  management.  As  an  example,  reference  is  made  to 
Figure  13  where  the  sensor  in  IR  band  2  dominated  in  terms  of  detection 
performance  over  most  of  the  range  of  interest.  This  condition  allows  one 


to  relax  the  requirement  of  using  the  radar  to  search  in  that  area  so  that 
the  radar  can  concentrate  its  efforts  in  another  region  where  the  IR  sys¬ 
tem  may  have  a  clutter  problem  or  severe  atmospheric  attenuation.  The 
radar  can  then  be  cued  by  the  IRST  to  provide  range  on  the  targets  in 
that  area  where  the  IR  system  is  providing  adequate  surveillance.  Hence, 
in  addition  to  optimal  thresholds  being  determined  from  the  multisensor 
adaptive  control  function,  information  is  obtained  which  can  be  useful  for 
optimally  managing  the  sensors  via  cueing  and  sensor  management  (i.e., 
dwell  time  and  field-of-view  management). 

3.0  CONCLUSIONS 

As  can  be  seen  from  Figures  12  and  13.  the  P^  enhancements  ob¬ 
tained  using  a  MSCFAR  are  limited  to  regions  where  at  least  two  sensors 
are  operating  with  relatively  similar  performance.  Note  that  this  similar 
performance  does  not  necessarily  mean  similar  sensitivity,  but  rather 
similar  P(j  which  is  related  to  a  variety  of  factors  (i.e.,  target  signature, 
target  fluctuation  statistics,  noise  statistics,  atmospheric  and  environ¬ 
mental  conditions,  and  sensor  sensitivity) .  All  of  these  factors  influence 
the  Pd  at  the  output  of  the  individual  sensor’s  threshold  detection  logic. 

The  primary  benefit  of  the  MSCFAR  is  the  information  that  it  pro¬ 
vides  about  each  individual  sensor’s  performance  in  a  particular  region.  If 
the  Pfg  chosen  for  a  particular  sensor  is  considerably  lower  than  the  Pf^ 
of  the  other  sensors  observing  the  same  angular  coverage,  MSCFAR  indi¬ 
cates  that  this  sensor  has  a  very  low  probability  of  detecting  the  target  in 
that  coverage  area  and  could  be  better  used  looking  in  an  area  for  which 
the  environmental  or  clutter  conditions  allow  the  sensor  to  work  at  its 
optimal  capability. 

What  has  been  presented  in  this  paper  is  a  first  cut  at  some  of  the 
issues  which  must  be  examined  for  optimal  performance  of  a  multisensor 
system.  The  examples  presented  consider  how  a  multisensor  constant 
false  alarm  rate  (MSCFAR)  processor  might  be  applied  to  the  case  of  a 
dual  band  IR  sensor  and  a  radar  for  optimal  detection  performance. 
Other  optimality  criteria  for  tracking  and  identification  need  to  be  devel¬ 
oped  for  effective  multifunction  operation  for  the  overall  multisensor 
suite. 
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Introduction 

This  note  examines  means  and  reasons  to  en¬ 
code  event  descriptions  into  bit  strings  -  or,  rather, 
strings  of  decision  flags  taking  the  values  TRUE, 
FALSE,  or  UNKNOWN.  In  the  case  of  perfect 
knowledge,  all  flags  are  TRUE  or  FALSE.  In  variants 
of  the  theory  where  degrees  of  confidence  are  taken 
into  account,  UNKNOWN  is  replaced  by  a  value  in  the 
interval  [0:1]  representing  the  degree  of  confidence 
that  the  TRUE  branch  of  the  decision  matches  ground 
truth  or  the  result  of  perfect  knowledge.  In  templates, 
a  fourth  value  -  ARB  -  may  be  used,  indicating  an 
event  attribute  the  same  as  or  related  to  attributes  of 
other  events  in  the  same  template. 

We  consider  here  the  encoding  of  events 
whose  place,  time,  and  other  attributes  are  known. 
Specifically  we  show  that  any  set  of  events  so  en¬ 
coded  can  be  sorted  to  facilitate  search  for  patterns 
such  as  troop  concentrations.  Templates  are  also  sets 
of  events  with  some  attributes  UNKNOWN  and  others 
ARB  in  the  sense  that  their  values  are  determined 
once  the  values  of  ARB’s  for  other  events  in  the 
template  are  known  from  matching  (unification)  with 
observed  events  in  a  history.  Thus  templates  contain¬ 
ing  an  ARB  are  properly  speaking  pattern  matching 
algorithms,  and  can  in  principle  be  complex  proce¬ 
dures.  A  match  with  an  UNKNOWN  attribute  in  a 
template  can  be  TRUE  or  FALSE  and,  unlike  ARB, 
does  not  propagate  a  value  into  other  attributes. 

The  possibility  of  linearly  ordering  the  events  in 
a  history  and  perhaps  in  a  template  also  enables  one 
to  search  an  event  file  for  aggregates  to  be  done  by 
passing  "windows"  over  the  event  file. 
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Decisions  About  Place  in  One  Dimension. 

The  one  dimensional  case,  though  quite  simple, 
introduces  a  useful  point  of  view  about  event  files. 

Consider  a  one  dimensional  Area  of  Interest 
(AOI),  being  simply  a  line  segement  whose  points  are 
identifed  by  the  real  numbers  in  the  half  open  interval 
[0:1 ).  Suppose  we  have  a  detection  procedure  able  to 
distinguish  events  in  [0:1/2)  from  those  in  [1/2:1 ).  If  an 
event  is  detected  in  the  first  subinterval  and  none  in 
the  second,  encode  it  by  a  0;  if  the  reverse  is  true  en¬ 
code  it  by  a  1 :  if  events  are  present  in  both  segments, 
mark  one  to  be  considered  later  and  encode  the  other, 
if  no  events  are  present,  record  the  presence  of  no 
events. 

Next  divide  the  Segment  containing  the  event  in 
two  equal  parts.  At  least  one  part  must  contain  an 
event;  if  both  contain  events,  mark  one  for  future  con¬ 
sideration,  and  encode  the  other.  Continue  until  the 
limit  of  detector  resolving  power  is  reached.  One 
event  will  have  been  identified  by  a  bit  string  record¬ 
ing  the  decisions,  and  others  may  be  marked  as 
having  been  partially  identified.  If  there  are  marked 
events  still  to  be  processed,  these  may  be  encoded 
by  continuing  the  decision  process  from  where  the 
nearest  was  saved  for  later  processing,  and  re¬ 
evaluating  the  part  of  the  decision  tree  descending 
from  the  save-node. 

This  decision  procedure  shows  how  to  locate 
and  count  events  in  a  one  dimensional  AOI,  and  il¬ 
lustrates  clearly  how  the  ability  to  count  is  limited  by 
the  ability  to  resolve,  provided  no  event  attributes 
other  than  spatial  ones  are  present.  It  should  be  noted 
that  the  concept  of  resolution  presented  here  differs 
somewhat  from  the  resolution  of  electromagnetic 
detectors  for  example,  because  there  is  an  unequivo- 
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cal  decision  about  location  of  events  even  when  they 
are  close  to  a  boundary  where  the  encoding  changes, 
but  which  would  not  be  clearly  recognised  by  a  real 
detector.  This  model  of  resolution  is  nevertheless 
good  enough  to  allow  analysis  of  the  effects  of  resolu¬ 
tion  on  INTEL  analysis. 

More  General  Encoding  of  Position  in  Space 
and  Time 

Consider  a  spatio-temporai  AOI  with  two  space 
dimensions  and  one  time  dimension,  i.e.  where 
events  take  place  in  a  two  dimensional  region  of 
space  (e.g.  UTM  zone  32U)  and  in  a  given  time  inter¬ 
val  (e.g.  the  calendar  year  1981  ).  The  two  space 
coordinates  x  and  y,  and  the  time  t  of  an  event  may 
each  be  encoded  by  binary  decision  as  outlined.  If  the 
three  codes  {x,y,tj  are  simply  concatenated,  we  have 
an  ordinary  Cartesian  coordinate  system.  A  more  use¬ 
ful  encoding  for  our  purposes  is  to  concatenate  the 
leading  bits  of  the  x,y,f  codes  into  a  tribit  code  show¬ 
ing  which  of  eight  subregions  in  the  AOI  contains  the 
event.  The  indicated  subregion  may  now  be  further 
divided  by  a  second  tribit  code  constructed  from  the 
next  most  significant  bits  of  the  coordinates.  This 
process  may  be  continued  until  the  precision  of  the 
data  is  completely  preserved  by  the  new  encoding, 
and  recorded  by  a  concatenation  of  the  tribit  codes  in 
the  order  they  were  decided,  early  decisions  being  in 
more  significant  positions  in  the  string.  In  practice  the 
subdivisions  of  the  AOI  are  rectangular  congruent 
tiles  because  this  simplifies  determination  of 
geographic  coordinates.  However  at  the  cost  of  more 
complex  coordinate  calculation  the  subdivisions  may 
be  made  in  any  way  we  please. 

These  encodings  have  useful  properties.  If  a  set 
of  events,  have  their  positions  encoded  in  this  way 
and  the  events  are  sorted  so  that  the  encodings  are 
in  odometer  order,  then  events  where  the  decisions 
about  spatio-temporal  position  diverge  only  late  in  the 
decision  tree  v^/ill  be  close  in  the  file  as  well  as  being 
close  in  space  and  time. 

The  converse  is  not  true  however.  Consider  a 
four  bit  encoding  of  position  on  a  line,  the  two  events 
(0111)  and  (1000)  are  in  neighbouring  small  seg¬ 
ments  of  the  line,  but  are  separated  at  the  first 
decision  because  one  is  at  the  midpoint  of  the  line, 
and  the  other  slightly  nearer  to  the  origin. 

In  spite  of  the  discontinuities,  whose  circumven¬ 
tion  will  be  discussed  later,  spatio-temporal  ag¬ 
gregates  such  as  troop  concentrations  not  crossing 
major  decision  boundaries  will  be  seen  as  high  den¬ 
sities  of  events  in  particular  regions  of  a  linear  file,  and 
thus  can  be  recognised  in  a  single  pass  through  the 
file  by  constructing  histograms  binned  according  to 
high  order  bit  patterns. 


The  search  algorithm  is  intrinsically  parallel  and 
fits  well  to  hypercube  architecture.  The  question  of 
other  paraliel  architectures  than  MIMD  has  not  been 
considered  properly,  but  there  are  no  clearly  visible 
difficulties  for  non  MIMD  machines. 

The  event  file  may  show  long-range  order  be¬ 
cause  for  example  an  aggregate  extending  across  the 
plane  x=1/2  in  a  coordinate  system  where  the  sides 
of  the  AOI  are  all  normalised  to  unit  length  will  give 
rise  to  two  peaks,  one  with  leading  bit  1 ,  and  the  other 
with  leading  bit  0.  The  way  the  signature  develops  can 
be  seen  most  easily  in  a  two  dimensional  4x4  ex¬ 
ample.  The  encodings  for  the  smallest  resolved 
regions  are  set  out  below  in  an  array  geometrically 
similar  to  the  physical  AOI. 

{0101)(0111)(1101)(1111) 

(0100)(0110)(1100)(1110) 

(0001  )(0011)(1 001  )(1  Oil) 
(0000)(0010)(1000)(1010) 

If  bins  in  a  histogram  are  numbered  by  the  in¬ 
tegers  corresponding  to  these  bit  encodings,  a  con¬ 
centration  in  the  four  center  squares  will  lead  to  peaks 
in  the  16  bin  histogram  at  bins  3,  6,  9,  and  12.  If  the 
concentration  is  in  the  upper  right,  it  will  be  seen  in 
bins  12,  13,  14,  and  15.  If  the  concentration  is  in  the 
right  half  of  the  region  but  centered  in  the  /  direction, 
the  peaks  will  be  found  in  bins  9,  11,  12,  and  14.  In 
each  case,  characteristic  peaks  will  be  seen  in  the 
histogram,  and  can  be  recognised.  The  rules  deter¬ 
mining  patterns  where  concentrations  cross  boun¬ 
daries  arise  quite  simply  from  bit  inversion  in  the 
encoding,  for  example  neighbouring  elements  across 
the  midpoint  of  a  one  dimensional  AOI  with  16  ele¬ 
ments  are  (0111)  and  (1000).  These  are  bitwise  com¬ 
plementary,  the  complementarity  being  a  signature 
for  the  particular  neighborhood  relation.  Once  a 
search  algorithm  has  recognised  a  possible  troop 
concentration,  the  representation  lends  itself  natural¬ 
ly  to  graphic  display  for  a  human  analyst. 

In  our  two  dimensional  case  the  complete  com¬ 
plementarity  is  seen  only  for  the  four  center  squares. 
The  second  case  i.e.  the  concentration  in  the  upper 
right  appears  as  quite  local  to  the  locations  (11AA) 
where  A  may  take  the  values  0  or  1 ,  and  the  third  case 
shows  complementarity  only  for  /  coordinates.  Thus 
if  a  histogram  of  events  is  made  at  some  resolution  of 
interest,  local  concentrations  will  seen  as  peaks,  and 
concerted  action  will  be  seen  as  long  range  correla¬ 
tion  in  bins  connected  by  the  complementarity  rela¬ 
tions. 
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Adjacency  and  Proximity 

Except  for  the  cases  connected  by  complemen¬ 
tarity,  proximity  of  two  events  is  characterised  by  coin¬ 
cidence  of  the  events  at  some  resolution  less  than  the 
maximimum  possible  resolution  in  the  representation, 
i.e.  by  descent  from  a  common  node  in  the  decision 
tree  determining  the  representation  of  position.  In  the 
case  of  complementarity,  nearby  events  show  com¬ 
plementary  paths  in  the  decision  trees  for  one  or  more 
coordinates,  and  identical  paths  in  the  others.  These 
relations  can  be  easily  tested  by  masking,  inversion, 
and  exclusive  OR  operations  on  bitstring  repre¬ 
sentations  of  position. 

Once  two  decision  paths  have  diverged  from 
regular  patterns  of  either  identity  or  complementarity, 
the  path  from  one  event  to  the  other  may  be  deter¬ 
mined  by  examination  of  the  low-order  bits  beyond 
the  point  of  separation,  and  the  distance  between  the 
two  events  determined.  Once  again,  apart  from  the 
foldings  of  the  decision  tree  onto  itself  induced  by 
complementarity,  leaves  close  in  space-time  are 
ciose  in  the  tree,  and  leaves  close  in  the  tree  are  al¬ 
ways  close  in  space-time. 

Other  Event  Attributes 

So  far  we  have  been  concerned  only  with 
spatio-temporal  attributes  of  events.  A  corresponding 
binary  classification  can  be  introduced  for  elements  of 
the  Table  of  Organisation  (units  in  the  chain  of  com¬ 
mand),  and  for  the  Table  of  Equipment  (pieces  of 
equipment).  Since  the  description  of  equipment  is  a 
Directed  Acyclic  Graph  (more  than  one  weapon  fires 
the  same  7.62  mm  ammunition)  the  decision  process 
is  a  little  different,  but  if  all  multiple  choice  decisions 
are  reduced  to  binary  ones,  bit  encodings  can  stiil  be 
used. 

There  are  still  questions  about  whether  the  en¬ 
codings  should  be  balanced  (i.e.  whether  they  are 
Shannon  optimal  and  arise  from  divisions  into  equal 
classes  at  each  stage),  or  whether  they  should  follow 
the  natural  structure  of  the  graph.  Since  the  natural 
encoding  can  always  be  derived  from  a  Shannon  en¬ 
coding  by  a  suitable  computation,  this  decision 
depends  on  a  tradeoff  between  use  of  CPU  time  and 
consumption  of  file  space.  The  translation  to  and  from 
optimal  Shannon  codes  can  always  be  hidden  from  a 
user  by  an  interface  to  the  database.  Thus  we  can  al¬ 
ways  assume  here  that  natural  encodings  are  used. 

In  summary,  a  complete  description  of  an  event 
is  either  the  concatenation  of  three  bit  strings,  one  a 
spatio-temporal  location,  one  a  description  of  the  unit 
and  its  state,  and  the  third  a  description  of  equipment 
and  its  state,  or  an  interleave  of  the  spatio-temporal 


string  of  tribits  with  bits  from  the  other  descriptors,  so 
that  the  event  descriptor  is  a  string  of  quinta-bits  (32- 
codes). 

The  32-codes  have  the  same  adjacency  and 
proximity  properties  as  the  position  codes  alone.  It  is 
not  clear  whether  the  equipment  and  command 
descriptors  shouid  simply  be  interleaved,  if  they  are, 
motorised  rifle  regiments  are  likely  to  well  separated 
in  the  file  from  spatially  nearby  associated  units  of 
other  types.  A  possible  compromise  would  be  to  inter¬ 
leave  equipment  and  command  descriptors  with  low- 
order  bits  in  the  spatio-temporal  descriptor,  so  that 
each  interleaving  takes  place  at  a  spatial  resolution 
appropriate  for  the  doctrinal  distances  belonging  with 
the  equipment  or  unit  in  question. 

The  question  of  using  these  kinds  of  techniques 
for  study  of  patterns  in  radio  communication  is  not  dis¬ 
cussed  here  except  to  note  that  a  bit  encoding  of  posi¬ 
tion  and  signal  parametrics  would  show  peaks 
corresponding  to  nets,  and  that  if  parametrics  were 
least  significant  it  might  be  possible  to  recognise  com¬ 
mon  nodes  of  two  nets. 

Templates  and  Matching 

Introduction  of  unit  and  equipment  attributes  to 
the  encoding  opens  new  questions.  It  becomes  pos¬ 
sible  to  have  items  in  the  event  database  for  units 
whose  command  attributes  are  known,  but  whose 
position  and  equipment  attributes  are  not,  and  vice 
versa.  For  example,  an  M107  175  mm  SP  gun  seen 
near  Dortmund  is  likely  to  belong  to  the  Artillery 
Division  of  the  1st  British  Corps  of  the  Army  of  the 
Rhine  (Isby  &  Kamps  1985),  but  without  knowledge  of 
unit  insignia  we  do  not  know  if  it  belongs  to  the  5th 
Regiment  or  the  32nd.  If  elements  of  the  5th.  Regi¬ 
ment  have  been  observed  much  further  away  than 
the  doctrinal  geographic  span  of  a  heavy  artillery  regi¬ 
ment,  there  is  presumptive  evidence  that  the  gun 
belongs  to  the  32nd. 

These  kinds  of  inference  are  familiar  in 
automated  reasoning,  the  simplest  matching  algo¬ 
rithm  is  called  unification  where  a  pair  of  events,  both 
with  unknown  elements  are  combined  with  a  rule 
saying  they  are  both  members  of  some  larger  class 
(Heavy  Artillery  in  the  1  st.  British  Corps  of  the  Army  of 
the  Rhine).  If  two  elements  are  both  members  of 
some  larger  class,  then  the  attributes  determining 
membership  of  that  class  must  be  the  same  for  each, 
and  attributes  at  lower  levels  may  be  mutually  ex¬ 
clusive. 

In  the  example  "Ml 07  SP  gun"  and  "Near 
Dortmund"  establish  membership  of  Heavy  Artillery 
Division  of  1st  Corps  of  BAOR.  Doctrine  and 
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geograpical  separation  establish  different  regiments, 
and  if  only  two  regiments  are  possible  and  one  is  else¬ 
where,  then  regiment  is  determined. 

This  process  propagates  knowledge  between 
observations  so  as  to  fill  gaps  using  properties  of 
higher  level  classifications.  One  natural  expression  of 
these  is  in  logic  programming  (PROLOG),  others  are 
in  paradigms  emphasising  properties  and  inheritance 
(SCHEME,  SMALLTALK,  LISP),  others  yet  are  the 
various  kinds  of  expert  system  shells.  Choice  of  the 
proper  paradigm  for  a  problem  can  have  great  in¬ 
fluence  on  speed  of  matching. 

Examples 

Moving  Target  Identification  and 

Stereoscopic  Vision 

These  are  two  similar  problems.  In  one  we  wish 
to  detect  differences  in  a  scene  as  perceived  by  two 
independent  sensors  displaced  laterally  relative  to 
one  another.  In  the  other  we  wish  to  detect  cases 
where  an  object  has  moved  since  it  was  last  ob¬ 
served.  In  both  cases  we  assume  the  object  has  al¬ 
ready  been  recognised  and  the  report  is  of  an  object 
of  some  type  at  some  place  and  time  in  the  case  of 
Moving  Target  Identification  (MTI),  and  of  views  of  an 
object  recognised  as  the  same  in  the  case  of  stereo¬ 
scopic  vision. 

Consider  the  MTI  problem.  A  single  pass 
through  the  event  file  can  eliminate  all  reports  of  the 
same  object  at  the  same  place  but  at  different  times 
by  an  exclusive  OR  and  mask  test.  If  it  is  desired  to 
restrict  attention  to  movement  in  some  interval  of 
time,  this  too  can  be  done  in  the  same  pass.  The 
resulting  set  of  events  will  have  remained  sorted  so 
as  to  maintain  the  proximity  properties  of  the  mapping 
from  space-time  into  the  selected  ordered  set.  Moving 
target  detection  is  done  in  two  phases.  After  a  resolu¬ 
tion  has  been  selected  appropriate  to  the  size  of 
movement  it  is  desired  to  detect,  a  window  is  passed 
across  the  set  to  record  all  cases  where  successive 
observations  of  the  same  object  are  close  together  in 
the  set  as  moving  targets,  and  then  remove  them.  Ail 
other  events  are  connected  to  each  other  by  com¬ 
plementarity,  connected  to  previously  recorded 
events  by  complementarity  or  are  observations  in 
error.  This  case  analysis  completes  the  moving  target 
detection. 

Stereoscopic  Vision 

There  are  few  major  differences  between 
stereoscopic  vision  and  Moving  Target  Indication,  ex¬ 
cept  that  stereoscopic  vision  often  requires  con¬ 
sideration  of  an  object  as  whole,  so  that  individual 
events  are  pixels  and  the  recognition  of  relative  dis¬ 


placement  in  a  stereo  pair  requires  comparison  of 
large  sets  of  events  each  representing  many  pixels.  It 
is  possible  to  run  a  cellular  automaton  as  a  parallel 
grid  problem  "compacting"  an  object  into  a  single 
pixel  so  as  to  define  the  position  of  the  object. 

The  question  still  needs  to  be  settled  by  experi¬ 
ment,  but  the  compaction  process  should  lead  to  posi¬ 
tions  differing  by  the  same  displacement  as  the 
original  objects.  If  there  is  significant  difference  be¬ 
tween  two  images  of  the  same  object  in  a  stereo¬ 
scopic  pair,  then  the  question  becomes  one  of 
defining  object  attributes  and  their  weights  so  as  to 
perform  a  feature  comparison  recognising  different 
views  of  the  same  object.  It  will  not  be  treated  here. 

The  Intelligence  Database 

Intelligence  data  encompasses  only  a  small 
part  of  ground  truth.  More  is  usually  known  about 
friendly  forces  and  intentions  than  about  the  enemy, 
but  substantial  inference  is  needed  to  derive  informa¬ 
tion  necessary  to  successful  operations.  This  in¬ 
ference  is  the  task  of  intelligence  analysis.  One  would 
like  to  provide  automated  aids  to  the  analyst  for 
operations  at  all  levels  from  tactical  company  and 
platoon  operations  to  the  strategic  planning  of  a  cam¬ 
paign.  The  representation  of  situation  as  a  set  of  con¬ 
cisely  encoded  events  is  necessary  to  support 
automated  analysis,  therefore  the  representation 
given  here  is  a  first  step  towards  this  goal. 

A  database  of  templates  can  also  be  repre¬ 
sented  as  histories,  and  with  the  availability  of  CD 
ROM  rotating  memory  devices,  very  large  databases 
can  be  provided,  even  for  systems  to  be  used  in  tac¬ 
tical  planning.  However,  because  plans  inevitably 
depend  on  terrain  as  well  as  disposition  of  troops,  it 
seems  unlikely  templates  can  be  predetermined 
these  will  depend  both  on  troops  and  terrain.  Terrain 
data  can  however  be  provided  on  CD  ROM,  and  dis¬ 
positions  of  friendly  and  enemy  troops  are  increasing¬ 
ly  obtainable  from  technical  intelligence  collection.  It 
therefore  seems  likely  that  templates  must  be  held  as 
simulation  algorithms,  using  a  representation  of 
knowledge  like  that  discussed  here,  and  parallel  com¬ 
puting  methods  to  achieve  satisfactory  performance 
in  real  time. 

Certain  problems  such  as  recognising  ag¬ 
gregates  of  various  kinds  and  simply  warning  a  com¬ 
mander  of  their  presence  seem  possible  today,  and 
we  suggest  ways  to  do  this.  The  next  generation  of 
commander’s  automated  assistants  should  be  able  to 
make  reasonably  sensible  command  decisions  and 
present  them  and  their  simulated  consequences  to  a 
commander  so  as  to  suggest  possibilities  that  might 
otherwise  have  gone  unnoticed.  This  may  be  more 
important  at  a  tactical  level  than  a  strategic  one  be- 
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cause  such  a  tool  should  help  inexperienced  troops 
do  well  in  combat,  and  may  be  important  in  the  air 
land  battle. 
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Introduction 

Intelligence  Analysis  is  inference  from  partial 
knowledge  of  the  real  world.  Two  commonly  held 
views  of  the  process  are  either  that  it  is  statistical, 
leading  to  computable  probabilities  of  truth  for  the 
various  conclusions,  or  that  it  is  a  theory  of 
preference,  leading  to  partial  ordering  of  degrees  of 
belief  in  the  conciusions. 

Both  views  have  strengths  and  weaknesses. 
The  preference  models  such  as  fuzzy  logic  have 
sound  foundations  (as  discussed  by  Goodman 
(1 987)).  They  represent  human  behaviour  in  uncer¬ 
tainty  well,  but  because  preference  models  order  con¬ 
clusions  only  on  the  basis  of  degrees  of  belief  in 
underlying  data,  they  do  not  quantify  questions  of  ex¬ 
pected  gains  or  losses  from  a  decision.  The  statistical 
models  are  able  to  deal  with  the  question  of  expected 
gain,  provided  the  probability  of  truth  of  elementary 
data  can  be  determined. 

This  paper  discusses  a  statistical  model.  It 
began  as  a  theory  of  intersections  between  sets  of  ob¬ 
served  events  and  sets  of  events  in  possible  state  se¬ 
quences  of  a  model  of  combat  (Griesel  1986,  Gabriel 
1986).  As  our  understanding  developed  we  realised 
that  a  central  issue  is  the  extent  of  knowledge  about 
possible  state  sequences  of  a  battle  or  campaign.  In 
an  ideal  situation  all  possible  sequences  are  known 
and  the  analysis  problem  is  simply  one  of  pattern 
matching.  In  practice  this  is  not  at  all  true,  one  has 
only  an  incomplete  database  of  templates  and  incom¬ 
plete  knowledge  of  state. 

Thus  a  full  theory  has  two  parts:  one  formulates 
the  ideal  model,  and  the  other  discusses  the  addition¬ 
al  difficulty  arising  from  incomplete  knowledge  of 
templates  and  its  contribution  to  Type  I  and  Type  II 
statistical  errors.  At  this  stage  it  seems  clear  that 
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results  of  the  second  part  of  a  theory  depend  on 
details  of  models  of  combat,  detection  processes  and 
the  like  as  well  as  the  approximations  necessary  to 
make  the  theory  usable  on  real  computer  systems. 

The  methods  to  perform  the  detailed  studies  of 
the  second  part  are  suggested,  but  owing  to  the  mag¬ 
nitude  of  the  necessary  effort  they  have  not  been 
used  except  for  a  simple  example.Rather  we  focus  on 
the  first  part  of  the  theory,  it  is  simple  except  that  in 
contrast  to  most  current  work,  it  involves  considera¬ 
tion  of  sets  of  events  extending  over  intervals  of 
space  and  time. 

Representation  of  Events 

Gabriel  and  Gabriel  (1988)  have  proposed  a 
representation  by  paths  through  a  binary  decision 
process  to  classify  an  event.  A  perfectly  known  event 
is  represented  by  a  bit  string,  where  each  bit  identifies 
whether  the  left  or  right  branch  of  the  decision  tree 
was  taken  at  some  node  corresponding  to  presence 
or  absence  of  a  property  of  the  event.  For  locations  of 
events  in  space  and  time  the  representation  is  a 
generalisation  of  a  quadcode.  lit  turns  out  that  96  bits 
locate  a  place  and  time  on  the  earth’s  surface  within 
a  fraction  of  an  inch,  and  to  better  than  a  second  in  a 
decade. 

Tables  of  Organisation  and  Equipment  can  also 
be  encoded  as  binary  decision  trees.  Roughly  speak¬ 
ing  22  bits  identify  a  soldier  or  equipment  item,  and  1 0 
more  encode  some  thousand  or  so  states  of  readi¬ 
ness  or  activity. 

Thus  an  event  in  this  model  is  presence  of  per¬ 
sonnel  or  equipment  in  a  given  interval  of  space  and 
time  and  in  a  given  state.  A  history,  or  scenario,  is  a 
set  of  these  events.  Gabriel  and  Gabriel  (1988)  show 
that  a  history  may  be  sorted  so  that  events  close 
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together  in  the  sorted  file  are  close  in  space  and  time. 
The  converse  is  not  quite  true,  however.  Some  events 
close  in  space  and  time  to  a  given  event  are  else¬ 
where  in  the  file,  but  in  regions  simply  related  to  the 
event  under  consideration.  The  event  file  may  be 
thought  of  as  marks  on  a  line.  Two  representations 
are  useful.  In  one,  all  the  attributes  of  time,  place,  unit, 
equipment,  and  readiness  are  encoded  into  position, 
and  the  mark  simply  indicates  existence  of  an  event. 
In  this  representation,  because  there  are  a  finite  num¬ 
ber  of  possible  events,  the  line  is  divided  into  the 
same  number  of  equal  segments.  Those  segments 
where  an  event  was  observed  are  marked  by  a  1 ,  the 
others  by  a  0.  This  "superencoding"  of  events  allows 
histories  to  be  represented  as  bit  strings.  It  is  not  used 
here. 

In  the  other,  time  and  place  are  encoded  as 
position  in  the  string,  and  equipment,  unit,  and  readi¬ 
ness  are  encoded  into  a  symbol.  A  blank  space  on  the 
line  is  also  considered  a  symbol.  In  this  case  the  his¬ 
tory  is  a  set  of  symbols  on  the  line.This  representation 
transforms  a  history  or  scenario  into  symbols  on  a 
line,  where  cause  and  effect  in  the  history  generate 
correlations  in  the  sequence  on  the  line.  The  decision 
process  for  position  in  space-time  recursively  sub¬ 
divides  the  Area  of  Interest  (AOI),  so  that  descent 
from  a  common  parent  node  in  the  decision  tree  im¬ 
plies  containment  in  a  region  of  space-time  defined  by 
the  common  parent.  This  containment  property  is 
preserved  on  the  line.  This  fact  seems  to  permit  fairly 
easy  recognition  of  aggregates  in  space. 

The  Intelligence  Database  and  System  Dynamics 

.  An  Intelligence  Database  is  a  history  of  obser¬ 
vations  of  the  past,  that  is,  a  set  of  events  each  being 
represented  by  a  path  through  a  classifying  decision 
tree.  It  is  incomplete  because  it  does  not  contain  all 
the  relevant  past;  for  example  details  of  enemy  plan¬ 
ning  for  the  campaign  are  likely  to  be  unknown.  Owing 
to  its  set  theoretic  foundation  it  is  well  matched  to 
modern  database  models  and  implementations.  The 
intelligence  Analyst’s  task  divides  in  two  parts:- 

1.  To  infer  more  knowledge  of  the  past  (i.e. 
ground  truth)  from  the  database  as  more  information 
is  received  about  the  present. 

2.  To  use  available  knowledge  of  ground  truth  to 
determine  enemy  intent  and  thus  to  predict  enemy  ac¬ 
tion. 

Success  in  both  of  these  depends  on 
knowledge  of  constraints  on  the  enemy  commander. 
These  are  primarily  terrain,  logistics,  doctrine  and 
weather.  All  of  them  prevent  completely  arbitrary  ac¬ 
tion  by  an  enemy,  and  therefore  they  introduce  cor¬ 


relations  in  the  history  over  space  and  time.  This 
introduces  correlation  in  the  symbols  on  the  linear 
representation  of  events.  This  correlation  is  the  only, 
evidence  of  enemy  intent,  provided  that  all  known 
events  are  represented  in  the  database. 

In  addition  to  being  incomplete,  the  intelligence 
database  is  also  contaminated  by  error.  It  can  be 
viewed  as  the  output  of  a  noisy  data  channel  from 
ground  truth  to  the  analyst.  Distinguishing  signals 
(knowledge  about  ground  truth),  from  noise  (am¬ 
biguity  about  meaning  of  incomplete  observations, 
and  deliberate  deception  by  the  enemy)  needs 
knowledge  of  the  possible  symbol  streams  that  might 
occur  in  ground  truth.  This  knowledge  allows  study  of 
the  collection  and  analysis  processes  by  the  methods 
of  Shannon  (1948),  and  determination  of  their  error 
rates  and  vulnerabilities. 

Using  the  Database 

If  the  bit  string  representation  of  events  is  used, 
there  is  a  finite  set  of  possible  events,  and  therefore 
of  possible  state  sequences.  These  are  an  ensemble , 
or  ideal  database  of  possible  past  histories  and  future 
developments  in  the  AOI.  This  ideal  database  of  pos¬ 
sibilities  is  called  an  ideal  template  database.  In  prac¬ 
tice  it  is  very  large  and  would  therefore  be 
implemented  as  a  set  of  algorithms,  which,  when 
given  an  observed  history  return  the  possible  ground 
truths  consistent  with  the  observations,  instead  of  as 
a  tabulation.  Practical  implementations  do  not  repre¬ 
sent  all  possibilities  completely  and  accurately,  but 
one  hopes  they  are  good  enough  for  the  purpose  at 
hand. 

The  Ideal  Template  Database 

Generation  by  Simulation 

In  the  ideal  template  database,  all  candidate 
scenarios  are  sets  of  events  containing  only 
TRUE/FALSE  binary  classification  flags.  If  enemy 
doctrine  were  well  known,  together  with  statistics  of 
equipment  failure,  commander’s  decisions,  and  the 
like  were  understood,  this  database  could  be 
generated  by  Monte  Carlo  simulation.  One  might  try 
to  generate  such  a  database  for  an  engagement  at 
the  platoon  level  as  a  demonstration  of  principle,  al¬ 
though  larger  scale  problems  need  more  resources 
than  should  be  spent  until  some  small  proofs  of  prin¬ 
ciple  have  been  obtained. 

Generation  by  Neural  Net  Algorithms 

It  also  may  be  possible  that  a  template 
database  could  be  generated  by  learning  in  a  neural 
network.  Possible  learning  algorithms  for  considera- 
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tion  are  Rumelhart’s  Back  Propagation  Algorithm 
(1986),  Kohonen’s  Feature  Mapping  (1986)ancl  pos¬ 
sibly  Josin's  Categorization  Model  (1977). 

The  Matching  Process 

In  the  ideal  template  database  all  events  con¬ 
tain  only  TRUE/FALSE  flags  and  the  database  is 
enormous,  but  the  matching  algorithm  is  simple.  If  the 
known  observations  are  found  in  the  past  of  a 
template,  to  within  the  error  of  measurement,  then  the 
future  in  that  template  is  a  prediction  of  a  possible  fu¬ 
ture  in  ground  truth.  A  formal  theory  of  measure  of  set 
Intersections  as  criteria  of  match  has  been  given  by 
Griesel  (1986). 

Since  templates  extend  from  the  past  into  the 
future,  even  if  the  past  is  perfectly  known,  it  will  match 
more  than  one  template,  the  different  futures  being 
determined  by  future  decisions,  and  other  stochastic 
phenomena.  In  addition,  since  the  past  is  usually 
known  only  imperfectly,  the  known  past  will  match 
templates  with  different  detailed  pasts.  Divide  these 
templates  into  groups,  each  group  containing 
templates  having  the  same  detailed  past.Different 
groups  have  different  pasts.  Either  the  present  state 
is  different  between  two  groups,  or  they  may  be  amal¬ 
gamated  because  the  differing  pasts  did  not  affect  the 
present  state.  This  divides  the  matching  templates 
into  groups  with  significant  differences  between  them. 


Thus  functionally  distinct  groups  of  possible 
past  and  future  event  sequences  differ  in  the  present, 
and  there  must  be  events  in  the  present  or  future  able 
to  distinguish  between  the  groups.  In  addition  unob¬ 
served  past  in  a  group  may  be  considered,  and  the 
probability  determined  that  it  would  have  been  unob¬ 
served  given  probabilities  of  detection  and  the  like.  As 
a  result  at  any  given  time  there  are  groups  of  possible 
templates  fitting  the  observations  of  the  past,  different 
groups  having  different  unobserved  pasts,  and  having 
various  computable  probabilities  of  representing 
complete  ground  truth. 

Evolution  in  Time  and  the  Commander’s  Task 

If  two  groups  have  the  same  history  in  the  past 
and  are  different  in  the  future,  there  must  be  a  split¬ 
ting  or  divergence  in  the  present  or  future.  At  the  point 
of  divergence  there  must  be  a  singularity  in  the  equa¬ 
tion  governing  evolution  of  history  with  time.  Recog¬ 
nising  these  singularities  seems  accessible  to 
spectral  theory  of  operators.  Clearly  decisions  by 
commanders  can  create  singularities,  in  fact  the 
commander’s  task  is  to  recognise  when  a  singularity 
can  be  generated,  and  take  action  to  drive  evolution 
along  the  most  advantageous  path. 


Measures  of  Performance 

The  template  database  is  an  ensemble  of  pos¬ 
sible  histories  or  scenarios.  It  can  be  assumed  that 
the  scenarios  are  equi-probable,  if  simulation  assigns 
different  probabilities  to  different  scenarios,  they  can 
be  replicated  in  the  ensemble  in  proportion  to  their 
probabilities. 

The  template  ensemble  can  be  sampled  to 
provide  a  Markov  source  of  possible  histories.  The 
samples  provide  a  symbol  stream  whose  rate  of  infor¬ 
mation  transmission  may  be  calculated  (Shannon 
1948).  A  collection  and  analysis  procedure  filters  the 
stream,  eventually  into  data  for  a  commander.  The 
commander  makes  decisions  based  on  the  analysis. 
In  a  simulation,  these  decisions  may  be  compared 
with  the  decisions  that  would  result  from  perfect 
knowledge  of  the  original  samples.  The  relative  costs 
and  gains  from  the  two  command  decision  sequences 
are  a  measure  of  performance  for  the  collection  and 
analysis. 

If  it  is  desired  to  evaluate  collection  and  analysis 
in  isolation  instead  of  by  their  usefulness  to  com¬ 
mander,  they  may  be  performed  on  a  history  up  to 
some  point  in  its’  evolution,  and  subsequent  evolution 
compared  with  prediction  by  the  analysis,  the  perfor¬ 
mance  of  the  INTEL  process  being  evaluated  by  the 
weighted  measure  of  the  intersection  between  events 
in  the  history  and  events  in  the  prediction  by  the 
analysis  algorithm,  the  weights  being  determined  by 
military  science. 

A  Simple  Example 

Consider  an  AOl  divided  into  a  64  by  64  array 
of  squares,  and  an  ensemble  defined  by  the  possible 
arrangements  of  identical  objects,  one  to  a  square  in 
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Fig.  1  Example  AOl 


the  AOl  (see  Fig.  1).  An  observation  of  one  object  by 
a  detector  able  to  resolve  coordinate  squares  yields 
12  bits.  The  resolution  of  the  detector  allows  all  pos¬ 
sible  configurations  to  be  distinguished,  and  the  cor¬ 
responding  measure  of  effectiveness  is  1 00%. 
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If  the  detector  resolution  deteriorates  by  a  fac¬ 
tor  of  two  in  the  x  and  y  directions,  each  observation 
will  yield  only  10  bits,  and  the  measure  of  effective¬ 
ness  is  25%  because  only  about  1024  of  the  4096 
possible  positions  of  an  object  can  be  distinguished. 

Consider  the  example  of  a  particular  configura¬ 
tion  of  objects,  roughly  representative  of  an  aggrega¬ 
tion  problem.  Suppose  four  objects,  disposed  as 
shown  in  Fig.  1 .  are  observed 

In  the  case  of  adequate  resolution  the  pattern  is 
recognised.  If  the  detector  resolution  is  halved  so  that 
a  hit  by  the  detector  represents  from  one  to  four  ob¬ 
jects  in  a  2  by  2  array  of  squares,  (as  illustrated  for  the 
lower  right  object  in  Fig.  1 .)  the  detector  will  report 
from  1  to  1 6  objects.  The  number  will  depend  on  how 
the  detector  randomly  places  2  by  2  squares  on  the 
pattern  so  that  each  square  contains  an  object,  and 
whether  a  hit  is  counted  as  one,  two,  three,  or  four  ob¬ 
jects  in  the  2  by  2  square,  and  so  on. 

It  is  perhaps  surprising  that  loss  of  detector 
resolution  by  only  a  factor  of  two  can  cause  such 
severe  uncertainties,  and  if  the  four  objects  were  far¬ 
ther  apart  or  closer  together  performance  would  be 
better.  On  the  other  hand,  the  configuration  shown  is 
typical  of  an  observation  early  in  the  development  of 
a  troop  concentration,  and  it  is  therefore  a  case  where 
good  performance  has  high  military  value. 

Automated  Analysis 

The  Processing  Hierarchy 

Automated  analysis  of  all  observations  by  one 
software  system  at  a  high  level  in  the  command  leads 
to  monolithic  software  and  hardware  architecture. 
Such  architectures  are  likely  to  be  difficult  to  change 
as  necessary  improvements  are  discovered,  and, 
owing  to  the  difficulty  of  the  software  development 
needed  in  the  first  place,  and  unlikely  to  be  properly 
responsive  to  real  needs.  A  possible  alternative  can 
be  based  on  present  manuai  techniques  as  discussed 
in  field  manuals  and  the  literature  on  C3I. 

The  essential  feature  of  these  manual  methods 
is  distributed  hierarchical  processing.  The  first  one  or 
two  generations  of  automated  systems  might  be 
automated  versions  of  the  manual  process.  To  design 
them,  a  theoretical  model  is  needed  allowing 
measures  of  performance  for  both  manual  and 
automated  technology. 

The  formal  model  is  a  command  tree  of  nodes 
connected  by  datapaths,  (i.e.  a  signal  flowgraph).  The 
data  and  material  flows  should  match  those  in  exist¬ 
ing  C3I.  Here  we  consider  only  INTELtasks,  and  rep¬ 


resent  both  data  and  material  flows  in  ground  truth  by 
dataflows  in  the  model. 

At  the  root  of  the  tree  is  the  highest  command, 
the  leaves  are  all  sensor  systems  and  personnel.  In¬ 
ternal  nodes  are  names  of  organisations  rather  than 
equipment  or  personnel,  that  is  to  say  the  subtree 
below  an  internal  node  describes  its  parts,  only  the 
leaves  being  physical  objects  such  as  people  and 
equipment.  In  complete  C3I  model  the  leaves  include 
manuever  and  logistic  personnel  and  equipment,  the 
dataflow  includes  command  and  control,  and  the 
decision  processes  include  command  decisions  as 
well  as  INTEL  tasking  and  interpretaion. 

Each  internal  node  of  the  tree  has  subordinate 
nodes  corresponding  to  the  command  structure.  This 
makes  the  branching  ratio  about  four  or  five  for  most 
nodes.  The  datapaths  correspond  to  human  com¬ 
prehension  and  communication,  and  thus  have 
roughly  the  same  capacity  at  each  level.  At  each  node 
there  are  two  decision  processes  and  a  local  memory. 


Incoming  data  passing  through  the  node 
towards  the  root  of  the  tree  (the  commander)  is 
divided  into  three  streams:  one  passed  on  up,  one 
passed  to  the  local  store,  and  one  discarded.  The 
local  store  is  purged  of  obsolete  data  so  as  to  make 
space  for  that  coming  in. 

For  dataflow  in  the  opposite  direction,  orders 
from  higher  nodes  are  augmented  by  information 
from  the  local  store  before  being  passed  lower  in  the 
command  structure.  The  data  compression  on  the 
way  up,  and  the  data  expansion  on  the  way  down  are 
done  by  two  decision  processes,  one  deciding  what 
to  send  on,  what  to  store,  and  what  to  discard,  the 
other  deciding  how  to  augment  incoming  orders  with 
data  from  the  local  store  so  that  they  can  be 
processed  satisfactorily  at  the  next  lower  level.The 
data  expansion  and  compression  ratios  are  the  same 
as  the  branching  ratios  for  the  tree,  as  they  must  be 
to  keep  the  traffic  along  the  datapaths  comprehen¬ 
sible  by  people  at  the  various  nodes. 

The  INTEL  decision  process  for  upwards 
travelling  data  recognises  patterns  in  incoming  infor¬ 
mation,  and  classifies  them  so  as  to  represent  the 
data  stream  by  sequences  drawn  from  a  limited  num¬ 
ber  of  symbols.  This  is  a  Shannon  (1948)  encoding 
into  a  stream  of  lower  capacity.  Unless  the  incoming 
stream  is  four  or  five  times  redundant  some  data  is 
lost  in  the  encoding. 

In  order  to  recover  the  essentials  of  the  situa¬ 
tion  at  the  lower  level,  enough  of  the  incoming  data  is 
recorded  to  allow  orders  travelling  downward  through 
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the  tree  to  be  expanded  so  that  they  can  be  carried 
out  by  lower  levels.  If  saved  patterns  recur  frequently 
they  become  permanent  templates  and  eventually  af¬ 
fect  doctrine.  Transient  data  Is  discarded  as  soon  as 
it  is  too  old  to  be  useful. 

There  is  probably  a  theory  of  decision  about 
transmission  and  storage  on  a  C3I  network 
analogous  to  the  theory  of  neural  nets,  but  the  com¬ 
plexity  of  the  decision  process  may  make  any  practi¬ 
cal  use  of  the  theory  impossible.  If  the  algorithms  for 
decision  and  data  storage  have  representations  by 
neural  nets,  then  the  model  is  a  heterogeneous  neural 
network. 

To  the  extent  that  the  most  important  data  is  dis¬ 
played  on  overlays  at  the  level  where  it  is  used,  it 
seems  quite  possible  that  the  symbols  at  each  level 
are  not  more  variable  than  could  be  described  by  ex¬ 
isting  icons  bound  to  coordinates,  with  a  theory  of 
recognition  and  value  based  on  unpublished  work  by 
Griesel  (1 986)  and  Gabriel  (1986),  using  the  data  rep¬ 
resentation  due  to  Gabriel  and  Gabriel  (1987). 

It  is  also  important  to  note  that  as  data  travels 
up  the  tree,  the  area  of  interest  increases,  but  the 
usable  spatial  resolution  changes  in  proportion  so 
that  algorithms  to  recognise  structured  aggregates 
are  unlikely  to  be  greatly  different  at  the  various 
levels. 

Defining  the  decision  processes  at  each  level 
by  explicit  models  will  be  a  formidable  task.  A  difficul¬ 
ty  in  doing  this  may  be  the  need  for  transfer  of  under¬ 
standing  in  both  directions  between  intelligence 
analysts  and  computer  scientists.  Proof  of  principle 
needs  to  be  established  by  experiment  on  a  small  well 
defined  problem  before  extensive  effort  is  considered. 


Once  quantitative  models  of  decision  proces¬ 
ses  are  made,  it  seems  a  laborious  but  straightfor¬ 
ward  task  to  determine  channel  bandwidths,  and  error 
probabilities  using  the  methods  of  Shannon 
(1948;reprint  1963). 

This  knowledge  is  important  in  planning  the 
next  generation  of  automated  systems:  it  gives  a 
quantitative  set  of  Measures  of  Performance,  allowing 
study  of  weaknesses  of  existing  systems  and  proper 
design  of  automated  replacements.  Relevant 
template  models  probably  exist  in  simulations  now  in 
use,  but  are  likely  to  need  refinement  and  validation 
before  they  are  sufficiently  reliable  for  use  in 
automated  analysis.  It  is  particularly  important  in  plan¬ 
ning  to  recognise  that  a  satisfactory  automated  sys¬ 
tem  need  not  be  perfect,  it  simply  must  do  better  than 
people  now  do. 


Conclusions 

There  exists  a  representation  of  events,  suited 
to  recognition  of  spatial  aggregates  such  as  con¬ 
centrations  of  troops.  Because  the  representation 
generates  strings  of  symbols  where  cause  and  effect 
relations  will  also  generate  correlations  in  the  symbol 
stream,  the  representation  is  well  adapted  to  dis¬ 
covery  of  causal  connections  by  a  variety  of  techni¬ 
ques  ranging  from  histograms  for  spatial  aggregates, 
through  parsers  for  aggregates  with  bounded  context, 
to  algorithms  for  story  understanding  in  the  case  of 
more  complex  situations. 

The  representation  allows  consideration  of  an 
ideal  template  database  as  an  ensemble  of  possible 
histories  for  a  campaign.  The  ensemble  model  leads 
to  an  information  theoretic  measure  of  performance 
for  INTEL,  and  probably  for  C3I  also.  Present 
template  implementations  typically  generate  subsets 
of  the  ideal  ensemble,  and  could  be  judged  by  com¬ 
parison  with  the  characteristics  of  the  ideal  database 
seen  as  a  Markov  source.  If  a  more  nearly  ideal 
template  database  can  be  generated  it  is  a  useful 
design  tool  even  the  generation  is  costly  and  slow. 

Real  template  databases,  particularly  those  for 
tactical  use,  may  be  imperfect  because  they  do  not 
contain  all  possibilities.  It  seems  impossible  to  know 
the  perfect  template  database  without  having  perfect 
simulation.  Actual  simulation  will  be  imperfect,  its'  im¬ 
perfections  can  only  be  found  by  comparison  with  ob¬ 
servation  of  behaviour  of  our  own  forces  and  those  of 
other  nations. 

An  approach  to  determination  of  a  nearly  per¬ 
fect  database  is  inclusion  of  patterns  seen  repeated¬ 
ly  in  the  past,  when  their  consequences  are  important. 
This  is  a  variant  on  the  processes  present  in  neural 
nets,  and  thus  a  possible  theory  of  learning  by  C3I 
systems  might  be  taken  from  theories  of  learning  by 
neural  nets  where  each  neuron  is  two  decision 
processes  and  a  quite  large  local  store.  In  this  case 
the  problem  of  predicting  changes  in  enemy  doctrine 
remains  open,  but  if  enemy  behaviour  is  seen  as  a 
Markov  source,  changes  in  behaviour  are  simply  un¬ 
predictable  changes  in  the  state  diagram  of  the 
source  and  the  only  response  is  to  recognise  them  as 
anomalies  relative  to  old  behaviour  as  early  as  pos¬ 
sible. 

Failures  in  this  recognition  are  well  documented 
for  the  events  of  August  1914  (Tuchman  1963)  and  of 
May  1940  Churchill  1949),  and  clearly  had  effects  on 
the  history  of  Europe  still  affecting  us  today. 

The  mooel  of  C3I  proposed  is  a  hierarchy  of 
decision  processes  and  local  memories.  This  model 
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matches  current  manual  systems  and  seems  likely  to 
have  scale-invariant  properties  arising  from  a  condi¬ 
tion  that  the  data  rate  of  communication  between 
nodes  be  roughly  the  same  at  all  levels.  This  mimics 
the  present  manual  systems  that  divide  incoming  data 
into  summaries  and  urgent  messages  to  be  passed 
up  the  tree,  and  other  data  to  be  kept  at  the  node  until 
it  is  obsolete  enough  to  be  safely  discarded.  It  seems 
likely  that  the  changes  of  spatial  and  temporal  scale 
as  data  passes  up  the  tree  might  allow  use  of  rough¬ 
ly  similar  algorithms  at  all  levels  to  recognise  interest¬ 
ing  structure.  Certainly  if  current  data  is  always 
displayed  as  icons  on  a  screen,  then  there  should  be 
similarities  between  pattern  recognition  at  the  bat¬ 
talion  level  and  at  the  division  level.  It  is  clearly  not 
necessary  for  C3I  automation  to  be  arranged  like  this, 
but  it  would  support  a  gradual  transition  from  manual 
methods  to  automated  ones,  and  would  allow  a 
limited  manual  capability  in  the  event  of  failure  of 
automated  systems  (except  loss  of  communication 
links). 

Finally  a  few  more  philosophical  remarks  may 
be  helpful  in  giving  perspective  about  the  problem. 
INTEL  analysis  is  like  scientific  research  in  the  sense 
that  both  are  efforts  to  predict  behaviour  of  the  real 
world  from  limited  observations.  The  significant  dif¬ 
ference  is  that  an  established  scientific  truth  does  not 
change,  although  better  measurements  may  refine  a 
theory.  In  contrast  understanding  of  an  enemy  may 
become  obsolete  because  of  deliberate  enemy 
decisions,  for  example  change  of  an  encryption,  or 
development  of  new  doctrine. 

The  history  of  science  may  provide  useful  warn¬ 
ings,  a  scale  for  the  possible  improvement  in  INTEL 
technology,  and  a  scale  for  its’  probable  difficulty. 

Secondly,  as  pointed  out  by  Shannon  (1949), 
the  theory  of  cryptanalysis  as  practiced  between  1 930 
and  perhaps  1 960  is  a  theory  of  extracting  signals 
from  noise,  and  at  least  for  the  rotor  machines,  a 
theory  of  semi-groups.  Simulations  evolving  in  time 
are  also  representations  of  continuous  semi-groups. 
Perhaps  some  of  the  unpublished  work,  particularly 
from  1940  on,  might  be  of  use  in  INTEL  analysis. 
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Our  interest  is  in  assessing  the  performance  of  a  multiple  correlation  hypothesis 
correlator/tracker  algorithm  (CTA)  in  a  false  alarm  environment.  Several  methods  of  analysis 
are  possible. 

At  one  extreme  are  empirical  approaches.  For  example,  one  might  model  the  behavior 
of  a  CTA  when  false  alarms  are  absent  (an  analytically  tractable  problem)  and  then  hope  to 
discover  some  form  of  empirically  derived  adjustment  or  scaling  law  to  extrapolate  to  the 
case  involving  false  alarms. 

At  the  other  end  of  the  spectrum,  one  might  attempt  to  develop  a  purely  analytic  model 
of  a  CTA,  involving  only  closed  form  expressions  or  possibly  recursive  relations.  Such  a 
model  could  then  be  used  in  parametric  studies  of  the  effects  of  false  alarms  on  CTA  performance. 
One  such  model  is  described  in  "An  Analytic  Model  for  the  Effect  of  False  Reports  on  Surveillance 
Tracking,"  by  Barry  Belkin,  and  Roberta  S.  Wenocur,  TechyUcat  PA,oczecUiig6  the,  19&7  TAt-SeAvtcz  Vata 
FLU>ton  Sympo^tam,  Vol ,  1,  June  1987,  pp.  631-640.  Unfortunately,  the  problem  of  modeling  CTA 
performance  analytically  is  a  very  difficult  one  and  such  models  require  a  great  many  simplifying 
assumptions  and  have  very  limited  predictive  capability. 

A  compromise  approach  is  a  hybrid  model  combining  analytic  expressions  for  tractable 
parts  of  the  problem  with  Monte  Carlo  simulation  on  a  very  limited  scale. 

We  briefly  survey  some  recent  results  based  on  the  three  types  of  approaches  described. 
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1.  INTRODUCTION 


In  multi-target  surveillance  problems, 
it  is  generally  not  possible  to  associate 
each  sensor  report  uniquely  with  its 
target  of  origin.  As  a  result,  there 
is  considerable  interest  in  probabilistic 
methods  for  report-to-track  correlation. 

One  class  of  such  data  correlation  methods 
is  based  on  the  processing  of  multiple 
report-to-track  correlation  hypotheses 
in  parallel.  For  example,  in  reference  1, 
multiple  hypothesis  techniques  referred 
to  as  Viack  iptitUng  and  optanaZ  Bay^Zan 
are  described.  The  appeal  of  multiple 
hypothesis  techniques  is  that  they  provide 
the  most  accurate  probabilistic  representa¬ 
tion  of  the  target  tracks  that  can  be 
extracted  from  the  sensor  data.  On 
the  other  hand,  because  of  the  tendency 
for  the  number  of  correlation  hypotheses 
to  grow  at  an  exponential  rate,  multiple 
hypothesis  methods  are  generally  quite 
computation  intensive. 

Continuing  advances  in  computer  design, 
however,  are  increasingly  favoring  the 
use  of  multiple  hypothesis  methods.  See, 
for  example,  reference  2  for  a  discussion 
of  a  multi-hypothesis  correlation  and 
tracking  algorithm  (CTA)  for  use  in 
the  midcourse  phase  of  the  Strategic 
Defense  Initiative  application. 

Our  present  interest  is  not  so  much 
in  multiple  hypothesis  CTAs  themselves, 
but  rather  in  predictive  low-level  models 
of  CTAs  which  serve  to  characterize 
CTA  performance  without  formally  carrying 
out  the  computation-intensive  CTA  functions. 
Such  models,  if  available,  would  find 
application  for  example  in  parametric 
studies  where  one  is  interested  in  investi¬ 
gating  system  performance  over  a  wide 
range  of  combinations  of  surveillance 
problem  parameters.  The  use  of  an  actual 
multi-hypothesis  CTA  to  carry  out  such 
parametric  studies  would  most  likely 
involve  a  prohibitive  amount  of  computation. 


A  second  application  of  low-level 
CTA  models  is  in  the  study  of  properties 
of  CTAs  which  are  particularly  difficult 
to  investigate  computationally.  For 
example,  one  might  be  interested  in 
characterizing  the  problem  conditions 
under  which  a  CTA  exhibits  a  well-defined 
steady  state  behavior.  It  might  require 
many  tracking  problem  replications, 
each  simulating  an  extended  period  of 
real-time  processing  to  obtain  statistically 
meaningful  computational  results  about 
the  steady  state.  A  low-level  model, 
on  the  other  hand,  might  provide  a  direct 
characterization  of  the  steady  state 
allowing  one  to  bypass  lengthy  computation. 

We  consider  low-level  predictive 
models  falling  into  three  general  categories 
analytic  models,  empirical  models,  and 
simplified  CTAs.  Analytic  models  allow 
one  to  predict  CTA  performance  from 
first  principles  based  on  closed-form 
mathematical  expressions  or  recursive 
relations.  Empirical  models  incorporate 
estimation  techniques  such  as  curve 
fitting  to  allow  one  to  extrapolate 
from  a  relatively  small  data  base  of 
representative  results  obtained  by  running 
an  actual  CTA  to  arbitrary  cases.  Other 
than  in  the  curve  fitting  phase  for 
an  empirical  model,  the  analytic  and 
empirical  methods  avoid  the  use  of  an 
actual  CTA  in  obtaining  predictions 
of  CTA  performance.  The  third  approach 
is  to  use  simplified  CTAs  directly.  Since 
the  generation,  processing,  and  pruning 
of  correlation  hypotheses  are  the  most 
computation-intensive  aspects  of  a  CTA, 
the  critical  simplification  of  such 
algorithms  involves  a  sharp  reduction 
in  the  number  of  such  hypotheses  that 
are  considered. 

In  Section  2,  we  develop  an  analytic 
model  for  the  special  case  in  which 
there  is  a  single  valid  target  and  no 
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false  targets;  i.e.,  the  false  report 
density  Xf  =  0.  We  will  cite  specific 
examples  of  each  type  of  low-level  model 
in  the  more  general  case  of  Xj’>0.  In 
Section  3,  we  will  present  numerical 
results  summarizing  a  study  of  the  predictive 
performance  of  one  particular  empirical 
model  based  on  a  so-called  ^Cjating  law 
and  one  particular  simplified  CTA. 

The  actual  CTA  used  in  this  comparative 
analysis  is  the  False  Alarm  Correlation 
Evaluation  Tool  (FACET) ,  developed  to 
study  the  correlation  and  tracking  problem 
in  an  environment  involving  a  single 
valid  target  and  a  moderate  to  high 
false  report  density.  Conclusions  are 
summarized  in  Section  4. 


2.  DISCUSSION  OF  MODELS  AND  PARAMETERS 
2.1  Low-level  models 

We  consider  each  of  the  three  broad 
categories  of  low-level  CTA  models  enumerated 
above . 

Analytic  models.  As  previously  noted, 
an  analytic  model  of  a  CTA  is  characterized 
by  the  availability  of  closed  form  mathe¬ 
matical  expressions  for  the  key  output 
quantities  measuring  the  degree  of  target 
localization.  Such  a  model  could  be 
used,  for  example,  in  parametric  studies 
of  the  effect  of  false  alarms  on  CTA 
performance.  An  analytic  CTA  model 
is  described  in  reference  3.  A  low-level 
model  for  a  multiple  hypothesis  CTA 
using  a  maneuvering  target  model  is 
postulated  and  an  analytic  expression 
is  derived  for  the  expected  surveillance 
area  that  must  be  searched  to  localize 
the  valid  target.  If  one  is  interested 
in  probability  distributions  rather 
than  expected  values,  the  problem  appears 
to  be  intractable.  In  the  special  case 
of  discrete  glimpse  opportunities  on 
a  single  valid  target  with  no  false 
alarms,  however,  under  appropriate  assump¬ 
tions,  one  can  derive  a  formula  for 
the  target  localization  distribution,  i.e., 
the  probability  of  target  containment 
as  a  function  of  area  searched. 

For  several  reasons,  this  baseline 
case  should  be  studied  before  introducing 
the  complications  of  false  alarms  into 
any  low-level  model.  First,  its  relative 
simplicity  allows  for  direct  analysis; 
second,  it  provides  guidelines  for  the 
more  challenging  situation  involving 
false  alarms;  third,  it  is  the  foundation 
for  models  which  extrapolate  or  scale 
from  the  no  false  alarm  case. 

We  select  as  our  performance  measure, 
the  valid  target  localization  distribution. 
Two  types  of  search  plans  are  considered: 
fixed  area  search  and  fixed  threshold 
search . 

Fixed  area  search.  Suppose  that  a 
fixed  search  area  A  is  assigned  and 
the  problem  is  to  determine  the  containment 
probability  Pc(A)  for  this  value  of 
area  and  for  false  alarm  density  Xp = 0 . 

In  this  case,  a  region  of  area  A  is 
chosen  to  maximize  the  probability  of 
target  containment. 


Our  approach  is  to  condition  first 
on  time  elapsed  since  the  last  detection 
of  the  valid  target.  For  this  purpose, 
we  must  consider  the  following  variables: 

T  =  time  between  successive 

discrete  glimpse  opportunities 

M  =  number  of  discrete  glimpse 

opportunities  since  most  recent 
detection  of  valid  target 

tp  =  sensor  report  processing 
delay  time 

S  =  (t-tp)  [mod  T]  . 

We  assume  that  evaluation  takes  place 
at  a  random  time  t  with  respect  to  the 
discrete  glimpse  process.  Thus,  time  t 
can  assume  real  values'  other  than  integer 
multiples  of  T.  Consequently,  S  is 
the  time  between  t  and  the  most  recent 
glimpse  opportunity.  The  total  elapsed 
time  since  the  last  detection  denoted 
Te  can  be  represented  as 

Te  =  M-T  +  S  +  tp. 

We  note  that  M,  t,  S,'  and  Te  are  random 
variables,  while  T  and  tp  are  parameters. 

Next ,  we  introduce 

(t)  =  Area  size  of  valid  target 
containment  region  at  time 
t  since  last  detection 

and 

a(t)  =  standard  deviation  of  the 

target  location  distribution 
at  time  t  since  last  detection. 

The  datum  growth  law  for  (t)  and  the 
formula  for  a(t)  depend  upon  the  state 
of  knowledge  of  initial  target  position 
and  velocity.  For  the  various  growth 
law  formulas,  see  reference  4. 

We  further  assume  that  the  target 
at  any  given  time  is  in  one  of  two  possible 
detection  states:  detectable  or  undetect¬ 
able,  with  rate  parameters  Xy  and  Xp 
associated  with  the  exponential  distribu¬ 
tions  for  time  spent  in  the  undetectable 
and  detectable  states,  respectively. 

For  each  fixed  value  m  of  M,  we  condition 
on  values  of  S.  We  assume  that  S  is 
uniformly  distributed  on  the  interval 
[0,T],  which  leads  to  the  conditional 
containment  probability 

Pc[A|M  =  m]  =.Y  Pr{.J!/(inT+s+tp)  <  A}  ds.  (1) 

Based  upon  properties  of  circular 
bivariate  Gaussian  distributions  and 
of  Integrated  Ornstein-Uhlenbeck  model 
(see  reference  4)  for  target  motion, 
the  integrand  in  equation  (1)  can  be 
expressed  as 

Pr{.^(tE)  <  A}  =  l-e-A/2ira2(tE),  (2) 

where 
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tE  =  mT+s+tp . 

To  condition  on  values  of  M,  we  define 


=  Pr{M  =  m}. 


Defining 


Q,  (t) _ i!L_  +-22_  e-(^D+xu)t 

Xd+Xu 


and 


^P  Xd+Xu  Xd+Xu 
we  can  now  compute  probabilities 
'Pq^\  and  P^^^  using  a  set  of  recursive 

relationships  as  follows,  where  Pp  is 
the  single  glimpse  valid  target  detection 
probability : 

Recursion: 


equidensity  contours  of  the  bivariate 
Gaussian  density. 

Let  Pmax^^p)  maximum  value 

attained  by  a  circular  bivariate  Gaussian 
density  with  variance  parameter  a2(tE) 
given  by 

Pmax^^p)  ”  7I”“27T~7  ' 

For  each  fixed  threshold  £,  one  deter¬ 
mines  which  values  tE  of  elapsed  time  Te 
are  meaningful.  If  tE  is  too  large, 
a(tE)  will  define  a  circular  bivariate 
Gaussian  density  whose  maximum  value 
PmaxC^E)  less  than  Z.  Conditioning 
on  S  and  M,  and  accounting  for  the  two-state 
Markov  detection  process  by  means  of 
Pm  one  may  compute  average  area  A(£) 
and  corresponding  average  containment 
probability  Pc(£) ,  for  the  given  threshold  A. 


p(n+l)  ^  p(n)Q^(X)  (1-Qq(T))  (1-Pp) 

p(n+l)^  p(n)  jx.Q^(T)  ]+P^")qq(T) 

and  ' 


To  determine  the  containment  area  A(£;tE) 
corresponding  to  fixed  target  density 
threshold  £  and  elapsed  time  tE,  set 
the  bivariate  circular  Gaussian  density 
equal  to  £  and  solve  for  A(£;tE)-  This 
yields 

A(£;tE)  -  -27Ta2(tE)ln(2iT£a2(tE))  .  (3) 


Initial  conditions: 


Then 


. _ ^ 

“  Xu+Xd 


Pj3.p{m)  . 


With  the  threshold  £  fixed,  and  for 
each  value  m  of  M,  we  condition  on  values 
of  S.  As  in  the  fixed  area  case,  we 
assume  that  S  is  uniformly  distributed 
on  the  interval  [0,T],  which  leads  to 
the  conditional  containment  area 

A(i|M  =  m)  =Y  jjA(iL;inT+s+tp)  ds  (4) 

and  corresponding  conditional  containment 
probability 

Pc(£  |M  =  m)  Pr{*j^(tE)  <  A(£;  tE)  )  ds  ,  (5) 

where  once  again 


Conditioning  on  values  m  of  M,  we 
obtain 

Pc(A)  =  I  PniPc[A|M  =  m], 
m=0 

with  Pc[A|M  =  m]  defined  by  equation  (1). 
It  is  interesting  to  note  that  when 
Xp  =  0,  the  target  is  continuously  detect¬ 
able  and  ^m  reduces  to 

Pn,  =  PD(l-PD)“. 


tE  =  mT+s+tp . 

By  substituting  from  equation  (3)  into 
equation  (2) ,  the  integrand  in  equation  (5) 
can  be  expressed  as 

Pr{«J^(tE)  <  A(£;tE)  }  =  l-27Ta2(tE)£, 

which  leads  to  a  simple  expression  for 
equation  (5)  for  each  form  that  a2(tE) 
assumes . 


that  is,  M  has  a  geometric  distribution 
as  one  would  expect. 

Fixed  threshold  search.  Instead  of 
assigning  a  fixed  search  area,  suppose 
we  set  a  fixed  target  density  threshold  £. 

In  principle,  this  means  that  search 
is  applied  to  regions  of  the  plane 
that  satisfy 

=  {  (x,y)  :  ftp(x,y)>£}  , 

where  ft£(x,y)  is  the  bivariate  Gaussian 

density  for  target  location  that  depends 
upon  the  datum  growth  time  tE-  These 
regions  are  in  fact  determined  by  elliptical 


Conditioning  on  values  of  M,  we  obtain 

A(«)  =  E  PBiAOl|M  =  m)  (6) 

m 

and 

Pc(«-)  =^Pn,Pc(^|M  =  m),  (7) 

where  the  sums  are  over  all  values  of 
m  that  are  meaningful  for  threshold  £, 
and  with  A(£|M  =  m)  and  Pc(£|M  =  m)  defined 
by  equation  (4)  and  equation  (5),  respec¬ 
tively. 

We  observe  that  one  can  show  that 
fixed  threshold  target  containment  prob¬ 
ability  curves  uniformly  dominate  those 
based  upon  fixed  area  search  (see  reference  5). 
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Unfortunately,  when  false  alarms  are 
present  and  multiple  correlation  hypotheses 
develop,  the  problem  of  determining 
the  target  containment  distribution 
appears  to  be  analytically  intractable. 

Empirical  models.  In  contrast  to 
an  analytic  model,  an  empirical  model 
uses  the  results  of  actually  running 
a  CTA.  For  example,  one  might  model 
the  behavior  of  a  CTA  when  false  alarms 
are  absent,  which  leads  to  an  analytically 
tractable  problem,  and  then  attempt 
to  discover  some  form  of  empirically 
derived  adjustment  or  ^cxiUng  law  to  extrapolate 
to  the  case  involving  false  alarms.  The 
basic  premise  of  the  scaling  law  in 
reference  6  is  that  any  surveillance 
scenario  including  false  contacts  is 
equivalent  in  terms  of  valid  target ^ 
localization  statistics  to  a  scenario 
without  false  contacts  and  with  a  lower 
valid  report  rate.  The  particular  adjust¬ 
ment  factor  is  derived  empirically  by 
curve  fitting  to  data  based  on  runs, 
of  a  CTA.  The  obvious  advantage  to 
a  scaling  law  is  that  once  the  empirical 
law  is  derived,  further  runs  of  a  CTA 
become  unnecessary. 

By  their  nature,  both  analytic  and 
empirical  models  do  not  allow  one  to 
study  the  computational  properties  of 
a  CTA.  Further,  a  scaling  law  provides 
no  information  on  the  region  of  stability 
(existence  of  a  steady  state)  of  a  CTA. 

Simplified  CTAs.  In  part  to  deal 
with  the  above  stated  limitations  of 
analytic  and  empirical  CTA  models,  the 
authors  have  developed  a  simplified 
CTA  that  combines  both  analytic  and 
Monte  Carlo  simulation  methods  to  predict 
CTA  performance.  Relative  scoring  of 
correlation  hypotheses  (-6cejae6)  is  based 
on  a  Bayesian  formulation  and  false 
alarms  are  simulated  using  a  spatial 
Poisson  process.  Current  scenes  condition 
on  past  sensor  report  assignments  only 
as  far  back  as  the  second  most  recent 
valid  report;  consequently,  fewer  hypo¬ 
theses  are  involved.  Type  I  and  Type  II 
errors  are  of  order  at  most  one,  where 
in  the  present  context  a  Type  I  error 
occurs  when  a  valid  report  is  declared 
false  and  a  Type  II  error  is  made  when 
a  false  report  is  declared  valid.  The 
inter-arrival  time  distribution  for 
valid  detections  is  derived  analytically 
by  assuming  the  same  two-state  target 
detectability  process  used  in  the  analytic 
model.  Spatial  overlap  in  the  conditional 
target  location  densities  is  ignored. 

This  simplified  CTA  model  reproduces 
many  of  the  operating  characteristics 
of  a  full-scale  CTA  but  at  a  higher 
computational  cost  than  analytic  and 
empirical  models. 

2.2  Regions  of  Parameter  Space 

One  can  think  of  surveillance  parameter 
space  as  being  subdivided  into  three 
general  regions: 


(1)  ReRion  I:  This  is  the  locus 
of  points  in  parameter  space 
where  the  false  alarm  rate  Xp 
is  low  and  the  predictive 
performance  of  iov7-level  models 
is  good  to  excellent. 

(2)  Region  2:  In  this  region, 

Xp  has  increased  to  the  point 
at  which  the  full  power  of 
multiple  hypothesis  CTA  logic 
is  being  invoked,  and  the 
predictive  capability  of  low- 
level  models  is  generally 
only  fair. 

(3)  Region  3:  This  is  the  region 
where  Xp  has  Increased  to 
the  point  that  the  CTA  is 
saturated  with  false  alarms, 
tracking  in  any  conventional 
sense  has  broken  down,  and 
low-level  models  have  no  useful 
predictive  value. 

It  should  be  pointed  out  that  in 
Region  3  one  could  not  reasonably  expect 
much  from  a  low-level  model  since  the 
CTA  being  modeled  is  itself  past  the 
point  of  providing  any  useful  tracking 
capability. 

An  important  issue  is  the  characteriza¬ 
tion  and,  in  fact,  the  proof  of  the 
existence  of  the  region  3  as  defined 
above.  One  might  reasonably  conjecture 
that  as  the  false  report  density  increases 
with  all  other  problem  parameters  remaining 
fixed,  the  performance  of  a  CTA  simply 
degrades  more  or  less  continuously  with 
no  sharp  transition  between  stable  and 
unstable  behavior.  Interestingly,  evidence 
that  this  is  not  the  case  is  reported 
in  Chapter  8  of  reference  1. 

Specifically,  the  authors  consider 
what  they  call  the  probabilistic  data 
association  filter  (PDAF) .  Under  the 
PDAF  method,  a  scan  of  sensor  reports, 
at  most  one  of  which  can  be  on  the  valid 
target,  is  processed  as  an  equivalent 
single  report.  Each  report  in  the  scan 
contributes  to  this  equivalent  report 
in  proportion  to  a  computed  likelihood 
that  the  given  report  is  actually  valid. 

The  as37mptotic  properties  of  the  PDAF 
are  investigated  by  numerically  iterating 
the  covariance  update  equation  (sometimes 
called  the  Ricatti  equation)  until  the 
filter  output  either  stabilizes  to  some 
type  of  steady  state,  or  exhibits  some 
form  of  instability,  i.e.,  the  iterates 
of  the  Ricatti  equation  do  not  converge 
to  an  identifiable  limit. 

Figure  8.2  of  reference  1  shows  a 
marked  boundary  between  the  region  of 
stability  of  the  PDAF  and  the  region 
of  instability.  Because  the  PDAF  is 
essentially  a  single  hypothesis  filter, 
the  existence  of  these  distinct  regions 
does  not  depend  on  a  choice  of  scene 
retention  threshold  settings  used  in 
multiple  correlation  hypothesis  pruning. 
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Consequently,  the  methods  of  reference  1 
can  be  used  to  investigate  how  a  CTA, 
even  if  operating  with  effectively  infinite 
computing  resources,  would  become  unstable 
if  the  false  report  density  were  high 
enough.  Unfortunately,  it  is  difficult 
to  investigate  this  instability  phenomenon 
using  a  true  multi-hypothesis  CTA  such 
as  FACET  since  an  apparent  instability 
may  occur  because  one  has  set  the  scene 
retention  thresholds  too  restrictively . 

We  observe  that  it  would  be  valuable 
to  identify  a  simple  quantity  that  measures 
the  intrinsic  difficulty  of  a  tracking 
problem  at  least  to  the  point  of  deter¬ 
mining  within  which  region  of  parameter 
space  a  given  problem  falls.  Once  one 
determines  the  appropriate  region  for 
a  tracking  problem,  one  can  decide  whether 
a  low-level  model  is  appropriate.  The 
scaling  law  described  in  reference  6 
implies  the  existence  of  one  dimensionless 
intrinsic  quantity  of  this  type.  It 
remains  a  subject  for  further  investiga¬ 
tion  whether  this  quantity  is  related 
to  the  boundaries  between  the  regions 
in  parameter  space. 


3 .  RESULTS 


Figures  1  and  2  show  a  direct  comparison 
of  FACET  output  with  that  of  the  analytic 
model  described  in  Section  2.  One  sees 
that  this  model  provides  an  excellent 
approximation  to  the  output  of  FACET 
in  the  case  of  no  false  alarms. 
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FIGURE  1 
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FIGURE  2 


Figures  3  through  5  compare  FACET 
output  with  that  of  the  scaling  law 
for  representative  parameter  values. 

In  Figure  3,  tracking  is  relatively 
easy  and  the  results  are  in  excellent 
agreement.  In  Figure  4,  tracking  is 
more  difficult  and  the  scaling  law  under¬ 
estimates,  for  a  given  search  area, 
the  probability  of  localizing  the  target. 

In  Figure  5,  tracking  becomes  even  more 
difficult  but  now  the  scaling  law  over¬ 
estimates  target  containment  probability. 
Additional  cases  of  FACET  have  been 
run  where  there  are  large  discrepancies 
between  FACET  and  the  scaling  law;  however, 
there  is  evidence  (reference  7)  that 
the  scene  retention  thresholds  in  FACET 
were  set  too  restrictively,  though  the 
problem  may  simply  be  in  Region  3  of 
parameter  space. 

Figures  6  through  8  show  a  comparison 
of  FACET  with  the  simplified  CTA  as 
described  in  Section  2.1  with  the  same 
parameter  choices  as  in  Figures  3  through  5. 
The  same  general  trend  of  tracking  of 
the  valid  target  becoming  more  difficult 
is  observed,  with  the  simplified  CTA 
more  consistently  overestimating  target 
containment  probability. 
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4.  CONCLUSIONS 


The  representative  CTA  FACET  and 
each  of  the  low-level  models:  the  scaling 
law,  the  analytic  model,  and  the  simplified 
CTA  agree  very  closely  in  their  estimates 
of  target  containment  probability  versus 
search  area  in  the  special  case  of  no 
false  alarms.  When  the  false  alarm 
density  Xp  is  small,  the  scaling  law 
and  simplified  CTA  models  continue  to 
agree  well  with  FACET.  The  problem 
of  finding  a  purely  analytic  model  in 
the  Xp > 0  case  appears  intractable.  As 
Xp  increases,  a  growing  disparity  develops 
between  FACET  and  the  low-level  models. 
Eventually,  for  Xp  sufficiently  large, 
tracking  breaks  down  totally.  Additional 
studies  are  needed  to  determine  the 
boundaries  of  the  regions  of  parameter 
space  where  low-level  models  show  funda¬ 
mentally  different  levels  of  predictive 
performance . 
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This  paper  addresses  the  multi-sensor,  multi-target  data  association  and  data  fusion 
problem  for  a  contact  management  function  in  a  submarine  combat  control  systems  envir¬ 
onment.  Only  geometric  mutual  data  association/fusion  is  considered  and  simulated 
results  presented.  Data  from  multiple  acoustic  sensors  are  associated  at  the  track 
level  using  a  ^  test.  The  sensor  data  can  be  fused  using  a  nonlinear  least  squares 
batch  processor;  or,  the  track  (solution)  data  can  be  fused  to  create  a  composite  track. 
The  paper  discusses  the  different  levels  of  data  fusion:  at  the  measurement  level  or 
the  track  level;  and,  presents  the  advantages  and  disadvantages  of  each  approach.  Error 
types  (Type  I  and  II  )  that  are  characteristic  of  an  association  decision  are  discussed  and 
experimental  (simulated  data)  results  are  presented  for  particular  scenarios  involving 
two  different  platforms.  The  simulation  results  show  that  Type  II  errors  (bad  asso 
iciaticns)  are  a  function  of  the  scenario  and  can  only  be  determined  experimentally  for 
each  scenario.  Even  manual  data  association  is  considered  to  determine  the  feasibility 
of  an  operator  being  able  to  decide  whether  two  different  contacts  should  be  associated. 
The  problem  of  fusing  track  level  data  that  is  produced  from  different  target  motion 
analysis  algorithms  is  examined.  Lastly,  since  the  work  presented  in  this  paper  is  of 
a  preliminary  nature,  data  association/fusion  concerns  are  discussed  and  topics  that 
need  to  be  addressed  in  more  detail  are  described. 
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INTRODUCTION 

Soviet  submarine  quieting  represents  the  single  most 
serious  threat  to  U.S.  Navy  superiority  since  World  War  II 
and  the  fleet  has  been  stating  for  years  that  no  single 
platform  or  Anti-Submarine  Warfare  (ASW)  tool  is 
capable  of  effectively  handling  this  threat.  The  quiet 
threat  mandates  improved  sensors  to  enhance  detection 
capabilities  and  will  require  an  integration  of  forces  for 
rapid  targeting.  These  improved  sensors  as  well  as  this 
greater  reliance  on  off-board  sensors  and  cuing  data  will 
inevitably  result  in  a  greater  number  of  contacts  which  in 
turn  will  present  data  overload  and  operability  problems  in 
ASW  Combat  Control  Systems.  Data  association,  the 
ability  to  determine  whether  multiple  contacts  are  from 
the  same  target  and  data  fusion,  the  ability  to  combine 
multiple  associated  contact  solutions  will  be  required  to 
present  a  single  concise  tactical  picture  to  all  friendly 
forces  in  multiple  target  environments.  Association  and 
fusion  can  reduce  ambiguities  and  increase  confidence  in 
the  tactical  picture,  can  enhance  spatial  resolution  utilizing 
the  large  apparent  aperture  between  cooperating  vessels, 
can  enhance  the  spatial  and  temporal  coverage  of  a  single 
vessel  and  can  enhance  detection  and  tracking  capabilities 
through  feedback  information. 

The  ASW  problem  can  be  difficult  due  to  the  poor  quality 
of  sonar  data  as  well  as  the  sparsity  of  the  data  as  contacts 
are  lost  and  reacquired  in  convergence  zones.  Thus,  poor 

observability,  often  characteristic  of  ASW  problems  can 
make  data  association  difficult,  and  the  quiet  threat  will 
further  compound  association  difficulties.  This  paper  will 
address  the  different  types  of  data  association,  the  levels 
of  data  association,  the  errors  which  can  occur  in 
association,  and  manual  association. 


DATA  ASSOCIATION  PRELIMINARIES 

The  first  level  of  any  data  association  routine  should  weed 
out  contacts  which  are  obviously  from  different  targets 
based  on  their  classification  and  attributes  as  well  as  their 
geographic  position.  Obviously,  it  is  unadvisable  to  try  to 
associate  a  surface  ship  with  a  submarine  if  one  is 
confident  about  each  contact’s  classification.  Likewise,  it 
is  ridiculous  to  try  to  associate  two  contacts  whose 
geographic  positions  are  significantly  different,  provided 
the  confidence  in  the  position  estimates  is  high.  Another 
way  of  prefiltering  association  candidates  is  based  on  a 
sensors  known  probability  of  detection  and  is  discussed  in 
Ref  1.  Simply  stated,  it  is  foolish  not  to  associate  two 
similar  contact  solutions  generated  from  sensors  with  high 
probabilities  of  detection,  since  the  probability  that  a 
second  target  was  present  and  undetected  is  low. 
Therefore,  the  contacts  most  probably  are  from  the  same 
target  and  the  estimates  can  be  fused  immediately.  Thus, 
an  association  test  should  be  carried  out  only  when  the 
source  of  the  two  contacts  is  doubtful.  Although  these 
preliminary  data  association  rules  are  heuristic,  they  could 
significantly  reduce  the  number  of  viable  association 
candidates,  could  prevent  erroneous  data  fusions  and 
could  prevent  missing  valid  data  fusions.  (See  Ref  2  for 
more  thoughts  on  data  association  preprocessing.)  The 
large  number  of  anticipated  contacts,  given  new  sensor 
technology,  will  require  prefiltering  the  association 
candidates  in  order  to  avoid  more  intensive  association 
processing  later. 

The  next  level  of  data  association  should  ensure  that  the 
state  estimates  and  covariance  matrices  were  generated 
using  independent  data  sets.  Performing  a  data 
association  test  on  the  outputs  of  two  algorithms  which 
have  used  common  data  will  most  likely  result  in  an  overly 
optimistic  composite  estimate.  The  status  of  contact 
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maneuvers  should  be  checked  to  ensure  that  no 
maneuvers  have  occurred.  A  contact  maneuver  will 
inevitably  produce  erroneous  correlations  between  the  two 
sensor  estimates  and  will  invalidate  the  data  association 
test.  Most  ASW  Target  Motion  Analysis  algorithms 
assume  a  constant  speed  and  course  target  motion  model 
and  rely  on  some  form  of  residual  test  to  detect  target 
maneuvers.  Unfortunately,  poor  scenario  geometry  and 
poor  measurement  quality  can  make  timely  detection  ‘of 
target  maneuvers  difficult  and  this  will  have  serious 
repercussions  in  data  association.  The  next  check  should 
ensure  that  the  contact  solutions  and  corresponding 
covariance  matrices  are  reasonable  and  that  the  algorithms 
computing  the  solutions  are  performing  as  expected.  The 
measurement  residuals,  can  be  used  as  a 

performance  indicator  as  suggested  in  Ref.  3.  One  quick 
check  which  can  be  made  to  ensure  algorithm  convergence 
is  to  compute  the  objective  function 

q(x)  ^(z-zVR-\z~z)  z^h(^)  (1) 

where  z  denotes  the  measurement  vector,  z  denotes  the 
measurements  which  would  be  observed  at  the  estimated 
state,  ;f,  and  R  denotes  the  covariance  of  the 
measurements 


and  compare  the  computed  value  of  the  objective  function 
to  its  expected  value.  The  objective  function  can  be 
shown  to  have  a  )C  distribution  with  m—n  degrees  of 
freedom  where  fti  is  the  number  of  measurements  and  n  is 
the  number  of  states.  Its  expected  value  can  be  computed: 


£[g(.v)l  = 


(3) 


estimates  which  apply  at  their  respective  times 
and  the  corresponding  covariance  matrices  Pi^Pj  and 
extrapolates  the  oldest  state  vector  and  covariance  matrix 
forward  in  time  to  the  most  recent  time  via  the  equations. 


,  Pi  =  if  ti  <  tj 


or 

Hj  =  ,  Pj  =  if  tj  <  ti 
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(5) 


The  X?  statistic  is  then  computed  as 


y^^(jti-ijf(Pi-^Pjr\^i-^j) 


(6) 


and  compared  to  a  threshold  S.  The  threshold  S  is  chosen 
from  a  ^  table  such  that  the  probability  thiU  ’)^>S  given 
the  assumption  that  Hi—ij  (contacts  trom  the  same 
target)  is  some  chosen  probability.  Thus  the  probability  ot 
failing  to  associate  two  contacts  which  are  actually  from 
the  same  target  can  be  set  a-priori.  If  ‘)^<S  then  the  two 
solutions  can  be  fused  via  the  equation 


^PAPr%+Pf^i\ 

where  Xf.  and  P^  ure  the  combined  solution  and  associated 
covariance  matrix.  If  then  the  solutions  should  not 

be  fused  and  the  data  streams  should  continue  to  be 
treated  as  separate  targets.  In  the  algorithm  implemented 
in  this  paper,  all  matrix  inverses  are  computed  using 
Cholesky  decomposition  via  the  equation 


A  quick  check  of  algorithm  performance  would  be  to 
compute  the  objective  function  and  compare  the 
computed  value  to  the  number  of  measurements  minus  the 
number  of  states.  If  the  objective  function  is  significantly 
larger  than  its  expected  value,  then  the  algorithm  is  not 
performing  as  expected  and  its  state  estimate  and 
corresponding  covariance  matrix  should  be  ignored  by  the 
data  association  routine.  This  quick  check  has  been  done 
for  both  a  batch  and  recursive  processor  and  has  proved 
to  be  a  reliable  performance  indicator.  Many  types  of 
performance  indicators  have  been  proposed,  but  the 
simple  test  previously  described  performs  well  and  is  not 
CPU  intensive.  Generated  target  solutions  should  always 
be  examined  before  use  in  any  kind  of  association  test, 
since  poor  algorithm  performance  can  result  in  some 
TMA  scenarios. 


GEOMETRIC  FUSION 

Once  all  of  the  initial  checks  on  the  association  candidates 
have  been  done  and  the  coordinate  systems  have  been 
aligned  to  ensure  gridlocking  (see  Ref  4),  a  simple 
statistical  test  such  as  a  test,  can  determine  whether  or 
not  two  contacts  are  from  the  same  source  (see  Ref  5). 
The  test  used  in  this  report  takes  two  independent  state 


A  =  L  U  (8) 

where  L  is  lower  triangular,  V  —  is  upper  triangular, 

I/-'  L'*  (9) 

and  17“^  is  computed  by  back  substitution. 

This  test  assumes  that  complete,  independent,  cartesian 
coordinate  solutions  (  x,y  position  and  velocity)  are 
available,  and  can  be  used  to  associate  and  fuse  only  two 
solutions.  However,  this  simple  algorithm  can  be 
modified  to  compute  the  most  likely  association 
candidates  and  could  be  extended  so  that  it  could  fuse 
multiple  solution  estimates. 

Although  the  test  assumes  complete  Cartesian 

Coordinate  state  estimates  and  covariance  matrices,  the 
test  can  be  used  to  associate  and  fuse  outputs  from 
various  algorithms.  The  TMA  algorithms  considered  in 
this  report  are  the  Non-I.inear  Least  Squares  algorithm 
(NLLS),  the  Modified  Polar  KAST  algorithm  (MPKAST) 
and  the  Modified  Spherical  KAST  algorithm  (MSKAST). 
The  NLLS  algorithm  discussed  in  this  report  is  a 
multisensor  Gauss-Newton  type  batch  processor  which 
employs  Householder  Orthogonal  'IViangularization 
techniques  and  was  developed  as  part  of  Raytheon’s 
Advanced  ASW  Combat  System  (see  Ref  6).  MPKAST 
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is  an  extended  Kalman  filter  which  processes  bearings  and 
range  measurements  and  is  currently  the  primary 
automatic  TMA  algorithm  in  CCS  MK  1.  MSKAST  is 
also  an  extended  Kalman  filter  developed  by  NUSC 
Newport  to  process  Wide  Aperture  Array  data.  Since  the 
NLLS  algorithm  is  formulated  in  Cartesian  coordinates 
and  since  both  KAST  algorithms  convert  their  polar  state 
estimates  and  covariance  matrices  into  their  Cartesian 
counterparts  via  appropriate  transformations,  it  is  simple 
to  fuse  data  from  these  algorithms,  Cartesian  coordinates 
were  chosen  for  the  data  association  test  since  the 
extrapolation  of  the  state  vector  and  covariance  matrix  is 
simple  and  faster  than  the  extrapolation  in  modified  polar 
or  modified  spherical  coordinates.  However,  a  little 
amount  of  shuffling  of  the  state  estimates  and  covariance 
matrices  is  needed  to  fuse  estimates  from  these  three 
algorithms  since  the  state  vectors  are  of  the  form 

NLLS  (x  V  i  V  z) 

MPKAST  (x  V  X  v) 

MSKAST  (x  V  z  >  v) 

It  is  also  possible  to  use  the  association/fusion  algorithms 
described  in  this  report  in  cases  where  the  number  of  state 
variables  in  the  association  candidates  differs.  For 
instance,  if  MPKAST  is  providing  estimates  of  target  x,y 
position  and  ;c,v  velocity  (calculated  from  the  modified 
polar  coordinates)  and  MSKAST  is  estimating  target  x,y 
position,  target  x,y  velocity  and  target  depth,  then  the 
depth  estimate  from  MSKAST  can  simply  be  ignored.  If 
depth  and/or  doppler  parameters  are  available  from  both 
candidate  algorithms,  the  association  routine  can  be 
modified  slightly  to  include  this  additional  information. 
To  handle  these  cases,  the  number  of  degrees  of  freedom 
in  the  ^  test  must  be  set  to  the  minimum  number  of  state 
variables  in  the  candidate  state  vectors,  and  the 
appropriate  threshold  must  be  used.  In  essence,  a  great 
deal  of  flexibility  exists  in  combining  solutions  from 
different  algorithms  processing  different  sensor  data. 

In  addition,  it  is  possible  to  modify  the  test  somewhat  to 
handle  localization  (position  only)  outputs.  If  only  target 
.Y,y  positions  are  estimated,  then  a  ^  test  with  only  2 
degrees  of  freedom  can  be  used.  However,  it  is  necessary 
for  the  localization  algorithms  to  provide  the  covariance 
matrices  associated  with  the  position  estimates,  which  are 
not  provided  in  any  current  combat  system.  The  towed 
array  crossfix  software  residing  in  CCS  MKl  been 
enhanced  by  Raytheon  so  that  localization  ellipses  can  be 
generated  along  with  the  position  estimates  for  any 
combination  of  azimuthal  bearings  and  conical  angles  from 
any  sensors  (Ref  7),  and  this  software  can  be  utilized  to 
try  data  association  on  position-only  outputs.  If  a  history 
of  contact  positions  are  available,  then  all  of  the  positions 
can  be  used  in  the  test  (as  is  done  in  Ref  1),  which  will 
result  in  a  more  reliable  test. 

In  some  cases  it  may  be  useful  to  attempt  a  data 
association  in  modified  polar  coordinates.  If  the  ownship 
is  restricted  to  a  single  leg  and  is  tracking  a  spherical  array 
contact  via  MPKAST  and  is  simultaneously  tracking  a 
towed  array  contact  via  MSKAST,  then  a  complete  TMA 
solution  is  unobservable,  since  range  is  unobservable  in  a 
bearings  only  or  conical  angles  only  case.  However,  since 
range  is  lacking  and  there  is  nothing  to  lose,  and  if  an 
operator  feels  he  has  the  same  target  on  both  sensors,  it  is 
possible  to  take  the  observable  quantities  from  MPKAST 
(i.e.,  bearing,  bearing  rate  and  normalized  range  rate  along 
with  the  corresponding  covariance  sub-matrix)  and  use  the 
bearing,  bearing  rate  and  normalized  range  rate  estimates 


along  with  the  corresponding  modified  polar  covariance 
matrix  from  MSKAST  (which  it  provides)  and  attempt  a 
data  association  on  the  three  observable  quantities.  If  the 
test  indicates  a  probable  correlation  between  the  contacts, 
then  the  data  sets  can  be  combined  to  yield  target  range. 
This  combination  can  take  place  at  either  the 
measurement  level  (via  NLLS)  or  at  the  ’’track"  level.  In  a 
first  leg  situation  like  this,  it  may  be  best  to  perform  the 
data  fusion  at  the  measurement  level  using  NLLS  with  the 
numerically  stable  Householder  technique.  Combining  at 
the  track  level  may  result  in  erroneous  solutions  due  to 
round-off  errors.  Once  the  data  fusion  has  been 
performed,  the  composite  solution  can  be  evaluated  using 
the  composite  objective  function  as  a  performance 
indicator  as  previously  discussed. 


GEOMETRIC  DATA  FUSION  LEVELS 

There  are  basically  two  levels  at  which  multi-sensor  or 
multi-platform  data  can  be  fused  once  two  contacts  have 
been  associated.  For  the  purpose  of  this  discussion,  it  will 
be  assumed  that  a  spherical  array  contact  and  towed  array 
contact  have  been  correlated  via  some  association  test. 
From  that  point  in  lime  on,  it  is  possible  to:  1)  continue 
to  treat  the  contacts  separately  and  fuse  data  at  the  track 
level,  after  generating  state  vectors  and  covariance 
matrices  for  both  the  sphere  and  towed  array  contacts  (see 
Fig.  1)  or  2)  fuse  data  at  the  measurement  level  and 
generate  the  single  combined  state  vector  and  covariance 
matrix  via  the  multi-sensor'  NLLS  algorithm.  Although 
fusing  data  at  the  different  levels  should  produce  identical 
results  assuming  no  round-off  errors,  data  fusion  done  at 
the  track  level  is  preferred  for  the  following  reasons: 

1.  this  approach  provides  the  operator  with 
information  on  the  constituent  solutions  (3 
localization  solutions  vice  only  1) 

2.  decorrelation  is  trivial  (already  done  since 
constituent  solutions  are  saved) 

3.  allows  for  future  statistical  tests  to  affirm/deny 
associations. 

4.  allows  for  error  budget  so  that  sensors  which  are 
contributing  little  to  a  solution  can  be 
reassigned. 


Figure  1.  Track/Measurement  Level  Fusion 
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The  disadvantages  of  track  level  data  fusion  are: 

1.  requires  more  CPU  time 

2.  requires  more  storage  (about  three  times  the 
storage  since  both  solutions  along  with  the 
combined  solution  is  saved) 

3.  less  numerical  stability  than  measurement  level 
fusion 

The  main  disadvantage  of  measurement  level  fusion  is  that 
measurements  are  usually  not  broadcast  between 
cooperating  vessels.  Usually,  only  solution  data  is 
transferred  making  measurement  level  fusion  impossible. 


In  general,  it  is  difficult  to  say  how  much  slower  track 
level  data  fusion  is  because  track  level  fusion  via  a  batch 
processor  may  require  many  iterations  to  generate  the 
individual  solutions  but  typically  only  a  few  iterations  are 
required  to  generate  the  combined  solution  (measurement 
level  fusion)  since  the  multi-sensor/multi-platform 
problems  are  more  observable  and  easier  to  solve.  The 
track-level  data  fusion  routine  presented  in  Eqns,  4-7, 
lakes  1/10  CPU  second  assuming  two  4-dimensional  state 
vectors  when  done  in  double  precision  on  a  VAX  11-780. 

In  order  to  demonstrate  the  equivalence  of  data  fusion 
performed  at  the  track  level  with  data  fusion  performed  at 
the  measurement  level,  the  NLLS  algorithm  was  run  in 
both  configurations  on  the  scenario  depicted  in  Figure  2. 
In  the  first  run,  the  NLLS  algorithm  was  executed  using 
ownship  bearings  from  target  1  and  the  state  estirnate  and 
covariance  matrix  at  the  end  of  the  scenario  were 
generated.  In  run  two,  the  NLLS  algorithm  was  executed 
using  only  consort  bearings  from  target  1  and  the  state 
estimate  and  covariance  matrix  at  the  end  of  this  run  were 
also  generated.  These  two  state  estimates  and^  two 
covariance  matrices  were  then  associated  and  fused  via  the 
algorithm  previously  discussed.  Then,  the  NLLS 
algorithm  was  executed  using  both  ownship  and  consort 
bearings  from  target  1  and  the  results  obtained  trom  this 
run  are  compared  to  the  results  obtained  by  fusing  at  the 
track  level  in  Table  1. 


All  of  the  NLLS  runs  presented  in  this  paper  have  been 
made  in  double  precision  and  employ  Householder 
Orthogonal  Triangulari/ation. 

Obviously,  the  state  estimates  and  estimated  solution 
quality  are  equivalent  except  for  small  roundoff  errors 
introduced  mainly  in  the  matrix  inversion  routines.  Data 
fusion  performed  at  the  measurement  level  will  result  in 
less  roundoff  errors  than  data  fusion  performed  at  the 
track  level  since  the  combined  information  matrix  is  apt  to 
be  less  singular  and  thus  more  easily  invertible  than  either 
of  the  constituent  information  matrices.  This  is  because 
the  TMA  problem  is  more  observable  and  easier  to  solve 
when  multiple  data  sets  are  used  simultaneously.  In  very 
poorly  observable  scenarios,  data  fusion  done  at  the  track 
level  may  suffer  heavily  from  roundoff  errors  and  the 
composite  estimate  and  covariance  matrix  may  be  biased. 
Therefore  for  long  range  low  bearing  rate  scenarios,  it  may 
only  be  feasible  to  fuse  data  at  the  measurement  level 
using  the  numerically  stable  Householder  Orthogonal 
Triangularization  approach,  since  the  individual  solutions 
may  not  be  computable  if  the  computer  word  length  is 
small. 

In  order  to  demonstrate  the  numerical  superiority  of  data 
fusion  performed  at  the  measurement  level  as  opposed  to 
data  fusion  performed  at  the  track  level,  the  NLLS 
algorithm  was  run  using  spherical  array  bearings  and 
towed  array  conical  angles  on  the  poorly  observable 
scenario  described  in  Figure  3.  The  NLLS  algorithm  was 
run  in  both  a  bearings  only  mode  and  a  conical  angles  only 
mode  and  the  two  solutions  and  covariance  matrices  were 
fused  at  the  track  level.  The  NLLS  algorithm  was  also  run 
in  a  spherical  array/towed  array  multi-sensor  mode  (data 
fusion  at  the  measurement  level)  and  the  results  are 
compared  to  the  track  level  fusion  in  Table  2. 


4  .  i  U»i 

10* 

10  KU 

SpliMi  Borini  Brroi 

Towtd  Ainy  8iu1d| 


i 

Fu.l<>n  •!  M«uur( 

y«»'  ^  Pu*!oa  »l  Tt»ek  L.« 

-  1410  Vi. 


;  «E  iko 


Figure  3.  Single  Platform  Scenario  Geometry 


Range 

Estimate 

Range 

St,  Dev. 
yds 

Bearing 

Estimate 

dug 

Bearing 
St.  Dev. 

deg 

Estimate 

deg 

Course 

St.  Dev. 

.  _ 

Estimate 

kts 

St.^Dcv. 

ktj 

Fusion  nt 

Track  Level 

62,919 

6,665 

91.824 

.120 

102.719 

22.773 

27.564 

■19.045 

Fusion  at 

Measurement 

Level 

57,509 

5,713 

91.933 

.110 

140.353 

219.103 

7.57-1 

24.102 

IDifferencct 

5410 

952 

.109 

.010 

37.634 

196.330 

19.99 

24.043 

Table  2.  Track/Measurement  Level  Fusion 
Single  Platform  Case 


Range 

Estimate 

yds 

Range 

St.  Dev. 

Bearing 

Estimate 

.. 

Bearing 
St.  Dev. 
deg 

Estimate 

deg 

St.  I>cv. 
deg 

Speed 

Estimate 

kts 

Speed 

St.  Dev. 
kts 

Fusion  at 

Track  Level 

61,136 

511 

94.004 

.138 

235.522 

3.191 

11.861 

1.363 

Fusion  at  | 

Me  as  Level 

61,150 

53S 

94.061 

,128 

235.449 

3.484 

11.830 

1,378 

IDilTerencel 

14  . 

27 

.003 

.008 

.072 

.293 

.025 

.015 

Table  1.  Track/Measuremenl  Level  Fusion 
Multi-  platform  Case 


(The  differences  in  the  90%  localization  ellipses  are 
depicted  in  Fig.  3).  Obviously,  the  fused  solutions  differ 
appreciably,  especially  course  and  speed  and  this 
difference  can  be  traced  to  the  large  roundoff  errors  which 
occur  when  inverting  poorly  conditioned  matrices.  When 
data  fusion  is  performed  at  the  track  level,  two  sets‘ot 
roundoff  errors  occur.  The  first  set  of  roundoff  errors 
occurs  when  the  covariance  matrices  for  both  the  sphere 
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and  lowed  array  are  inverted.  Then  the  track  level  fusion 
routine  inverts  the  sum  of  the  information  matrices  adding 
on  even  more  roundoff  errors.  Obviously,  these  errors  can 
be  significant  even  when  using  double  precision  arithmetic. 

DATA  ASSOCIATION  ERROR  TYPES 

There  are  two  types  of  errors  which  can  occur  in  data 
association.  Type  I  errors  are  made  when  the  null 
hypothesis  (i.e.,  the  contacts  are  from  the  same  target)  is 
incorrectly  rejected  or  stated  differently,  when  the 
algorithm  fails  to  combine  contacts  which  are  actually 
from  the  same  target.  The  consequences  of  Type  I  errors 
are  that  two  poor  solutions  from  supposedly  two  different 
targets  are  maintained  when  one  potentially  good  solution 
is  possible.  Type  II  errors  occur  when  the  null  hypothesis 
is  incorrectly  accepted,  or,  when  the  algorithm  combines 
two  contacts  which  are  from  different  targets.  The 
consequences  of  Type  II  errors  are  that  a  poor  solution 
along  with  an  overly  optimistic  solution  quality  can  be 
generated.  Type  II  errors  are  more  serious  than  Type  I 
errors  since  they  may  indicate  a  false  position  with  a  tight 
error  bound. 

In  order  to  demonstrate  the  consequences  of  both  types 
of  error,  the  TMA  scenario  depicted  in  Fig.  2  was 
simulated.  Ownship,  "O/S",  is  processing  spherical  array 
bearings  Avith  a  bearing  standard  deviation  of  2®  into  an 
idealistic  20  second  block  averager  and  has  made  four  5 
minute  legs  with  60"  lead/lags  at  a  si>eed  of  10  knots.  The 
consort,  "CST”,  is  also  processing  spherical  array  bearings 
with  a  bearing  standard  deviation  of  2"  into  the  same 
block  averager  and  has  made  roughly  3  seven  minute  legs 
and  is  also  doing  60"  lead/lags  at  12  knots.  Figure  4 
shows  the  90%  localization  ellipses  on  target  1  as  seen  by 
the  ownship  and  consort,  along  with  the  fused  solution.  If 
the  algorithm  failed  to  associate  the  two  contacts,  then 
both  poor  solutions  would  be  maintained  on  the  "two" 
targets  rather  than  have  the  one  high  quality  solution. 
Type  I  errors  will  result  in  the  loss  of  potentially  high 
quality  solutions  which  could  lead  to  missed  or 
unnecessary  weapons  launches  and  will  necessitate  storing 
and  processing  redundant  data  sets. 

Figure  5  graphically  depicts  the  gravity  of  Type  II  errors. 
In  this  scenario,  ownship  has  processed  the  bearing 
measurements  from  target  1  while  the  consort  has 
processed  bearing  measurements  from  target  2.  The 
localization  ellipses  generated  by  both  ownship  and 
consort  are  depicted  along  with  the  fused  solution. 
Obviously,  the  fused  solution  is  off  significantly  and  the 
estimated  quality  of  the  fused  solution  is  excellent. 
Therefore,  this  type  of  error  could  lead  to  a  useless 
weapons  launch  at  a  false  target  position  where  the 
probability  of  hitting  either  one  of  the  targets  would  be 
slim.  False  associations  will  lead  to  very  misleading  tactical 
pictures. 


Figure  4.  Missed  Association 


Figure  5.  Invalid  Association 


It  is  wise  to  avoid  both  types  of  error,  however,  any 
association  algorithm  will  exhibit  both  types  of  errors  to 
some  degree  and  a  good  data  association  algorithm  should 
provide  the  tools  to  rapidly  associate  two  contacts  or 
gracefully  de-associate  one  supposed  target  into  its 
constituent  contacts,  since  data  association  errors  are 
inevitable.  When  the  test  is  used,  it  is  possible  to 
specify  the  probability  of  the  Type  I  errors  by  setting  the 
threshold  used  in  the  test,  however,  it  is  impossible  to 
directly  specify  the  probability  of  the  more  serious  Type  II 
errors  because  the  conditional  probability  that  is  greater 
or  less  than  a  threshold  given  that  the  two  contacts  are  not 
from  the  same  target  can  not  be  calculated,  unlike  the 
classical  signal  detection  problem.  Unfortunately,  setting 
the  ^  threshold  sets  the  probability  of  both  types  of  error, 
and  the  probability  of  Type  II  errors  will  be  a  function  of 
the  scenario.  Therefore,  the  attribute  comparisons 
between  contacts  will  play  a  key  role  in  minimizing  both 
types  of  error  and  its  importance  can  not  be 
underestimated.  In  many  ASW  applications,  good 
classification  data  on  surface  vessels  can  be  accompanied 
by  poor  positional  data  and  the  classification  data  must  be 
fully  exploited. 

The  threshold  used  in  the  association  test  may  be  dictated 
by  both  the  number  of  contacts  and  the  tactical 
circumstances  and  rather  than  propose  a  threshold  for  the 
test,  the  relationship  of  the  threshold  to  both  types  of 
error  will  be  examined. 


EXPERIMENTAL  DATA  ASSOCIATION  RESULTS 

In  order  to  examine  the  performance  of  the  data 
association  test,  400  simulations  of  the  scenario  depicted 
in  Fig.  2  were  run  where  both  the  ownship  and  consort 
were  tracking  target  1.  In  each  simulation,  the  contacts 
are  treated  separately  and  data  association  and  fusion  was 
performed  at  the  track  level.  Various  probabilities  of 
missed  associations  were  chosen  and  the  results  are 
presented  in  the  following  table. 


Probability 
of  Missed 
Associations 

Threshold 

Actual  %  of 
Correctly  Associated 
Runs 

Actual  %  of 
Missed 
Associations 

.50 

3.375 

50% 

50% 

.20 

5.989 

74% 

26% 

.10 

7.779 

81% 

19%  . 

.05 

9.488 

87% 

13% 

.01 

13.277 

91% 

9% 

Table  3.  Multi-platform  Fusion  Results 
Missed  Associations 
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A  perfect  data  association  routine  would  associate  all  of 
the  solution  pairs,  and  if  the  threshold  was  set  to 
infinity,  the  algorithm  would  indeed  associate  all  solution 
pairs.  Unfortunately  the  algorithm  will  associate  all 
solution  pairs  whether  or  not  they  are  from  the  same 
target.  Therefore,  as  in  any  kind  of  association  test,  the 
chosen  threshold  is  a  compromise  based  on  the 
consequences  of  both  types  of  errors.  As  can  be  seen 
from  the  table,  the  number  of  correctly  associated  runs  is 
low  for  high  probabilities  of  missed  associations.  An 
unusually  large  number  of  high  ^  values  were  found  and 
38  runs  exceeded  the  threshold  of  13.377  when  only  4  runs 
over  this  threshold  would  be  expected.  Four  runs 
produced  statistics  of  over  100,  which  is  extremely 
improbable.  Although  the  reason  for  both  the  low  number 
of  correct  associations  and  large  number  of  high 
values  is  not  presently  known,  it  is  possible  that  the 
non-Ciaussian  nature  of  the  target  solutions  is  to  blame. 
In  non-linear  applications,  the  distributions  of  the 
estimated  state  variables  are  not  normal,  which  the  yc 
test  assumes,  'fhis  problem  is  currently  being  investigated. 

In  order  to  determine  the  frequency  of  type  II  errors  (bad 
associations),  400  simulations  of  the  same  scenario  (see 
Fig.  2)  were  run  where  the  owns  hip  is  tracking  target  I  and 
the  consort  is  tracking  target  2.  The  same  probabilities  ot 
type  I  error  (missed  associations)  as  in  the  previous  table 
were  used  and  the  results  are  presented  in  the  following 
table: 


generated  by  both  the  ownship  and  its  consort  (see  Figure 
6)  and  it  is  apparent  that  the  two  targets  are  easily 
distinguishable  since  their  90%  localization  ellipses  are 
disjoint.  In  general,  making  better  measurements  or 
collecting  more  data  will  substantially  aid  in  association 
testing  and  it  is  important  to  keep  testing  to  affirm  or  deny 
any  previous  associations,  especially  when  measurement 
data  is  poor  and  the  associations  are  weak.  To 
demonstrate  this,  data  association  was  tried  on  400  runs 
for  the  case  where  both  friendly  vehicles  are  tracking 
target  1  using  various  probabilities  of  missed  associations. 
When  both  friendly  vehicles  are  tracking  target  1,  the  data 
association  lest  performs  as  expected  as  shown  in  the 
following  table: 


Figure  6.  Multi-platform  Scenario  Geometry 
High  SNR  Case 


Probability  of 
Missed  Associations 

■>d 

Threshold 

Actual  %  of 

False  Associations 

.50 

3.375 

4% 

.20 

5-989 

36% 

.10 

7.779 

66% 

.05 

9.488 

83% 

.01 

13.277 

94% 

Table  4.  Multi-platform  Fusion  Results 
Incorrect  Associations 


As  anticipated,  lowering  the  probability  ot  missed 
associations  results  in  an  increase  in  the  probability  ot 
false  associations,  since  it  is  impossible  to  decrease  the 
probabilities  of  both  types  of  error  simultaneously.  In  this 
scenario,  if  the  probability  of  missing  an  association  is 
chosen  to  be  .10,  then  given  the  track  of  the  second  target 
’TCT^2",  there  is  a  66%  probability  that  the  two  targets 
will  be  incorrectly  fused,  resulting  in  a  poor  solution  and 
tight  error  bound  (see  Fig.  5).  If  a  different  target  2  track 
was  specified,  then  the  number  ot  lalse  associations  might 
be  reduced  significantly,  which  demonstrates  the  point 
that  the  probabilities  of  missed  associations  can  be  set; 
but,  the  probabilities  of  false  associations  are  scenario 
dependent  and  can  only  be  determined  experimentally.  If 
the  second  target  track  was  chosen  so  that  it  was  not  even 
close  to  the  first  target  or  if  the  solution  quality  was  high, 
then  very  few  false  associations  would  occur.  When  two 
targets  are  close  to  each  other  and  traveling  in  the  same 
general  direction,  and  when  the  solution  quality  of  both 
targets  is  poor,  then  the  probability  of  false  associations 
will  be  high.  Therefore,  even  specifying  a  probability  of  a 
missed  association  is  difficult. 

In  order  to  show  how  drastically  the  TMA  scenario 
parameters  affect  data  association,  the  scenario  described 
in  Fig.  2  was  repeated,  however,  the  measurement  noise 
was  reduced.  Reducing  the  measurement  noise 
substantially  increases  the  quality  of  the  solutions 


Probability 
of  Missed 
Associations 

Threshold 

Actual  %  of 
Correctly 
Associated  Runs 

Actual  %  of 
False 

Associations 

.50 

3.375 

53% 

0% 

.20 

5.989 

83% 

0% 

.10 

7.779 

91% 

0% 

.05 

9.488 

95% 

0% 

.01 

13.277 

99% 

0% 

Table  5.  Multi-platform  Fusion  Results 
High  SNR  Case 


Using  the  same  probabilities  of  missed  associations,  the 
data  association  routine  did  not  falsely  associate  any 
solution  pairs  for  the  case  where  ownship  was  tracking 
target  1  and  the  consort  was  tracking  target  2.  In  the 
previous  example  (larger  measurement  noise)  there  were  a 
great  deal  of  false  associations  since  the  solution  quality 
was  poor.  In  general,  poor  measurement  quality  and  poor 
scenario  geometry  will  result  in  both  poor  TMA  solutions 
and  poor  data  association  results. 


MANUAL  DATA  ASSOCIATION 

It  is  interesting  to  see  what  happens  to  the  bearing 
residuals  when  contacts  are  correctly  and  incorrectly 
fused,  since  a  manual  association  decision  may  be  based 
on  the  current  dot  stacking  technique  used  in  today’s 
Combat  Control  systems.  Figure  7  shows  the  bearing 
residuals  as  seen  by  both  the  ownship  and  consort  for  the 
scenario  depicted  in  Fig.  2.  The  leftmost  plot  shows  the 
bearing  residuals  produced  by  taking  the  difference  of  the 
ownship’s  measured  bearings  and  the  predicted  bearings 
based  on  the  ownship  solution  and  the  bearing  residuals 
produced  by  taking  the  difference  of  the  consort’s 
measured  bearings  and  the  predicted  bearings  based  on 
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the  consort’s  solution.  The  middle  plot  shows  the  bearing 
residuals  produced  by  taking  the  difference  of  both  the 
ownship’s  and  consort’s  measured  bearings  and  the 
predicted  bearings  based  on  the  combined  solution  for  the 
case  where  the  valid  association  is  made  (see  Fig.  4).  It  is 
interesting  to  note  that  although  the  solution  improves 
significantly  when  the  contact  solutions  are  validly  fused, 
the  objective  function  increases  slightly  for  both  ownship 
and  consort  goes  from  69.42  to  69.62  while  0^/  goes 
from  44.27  to  45.41).  Although  this  seems  contradictory, 
it  is  easily  explainable.  For  the  no  noise  case,  the 
minimum  of  the  objective  functions  will  be  zero  and  will 
occur  at  the  true  target  state  (see  Figure  8).  When  noise 
is  added,  ownship  will  minimize  its  objective  function  and 
obtain  its  minimum  value  Q^s  at  its  best  estimate 
Similarly,  the  consort  will  minimize  its  objective  function 
and  obtain  its  minimum  value  Qcst  at  its  best  estimate 
When  the  states  are  fused,  and  the  objective 
functions  are  computed  using  the  fused  state  estimate 
(which  is  better  than  either  single  estimate),  the  objective 
functions  and  0^5/  niust  increase  since  their  minimum 
values  occur  at  and  respectively.  The  rightmost 
plot  of  Figure  7  shows  the  bearing  residuals  produced  by 
taking  the  difference  of  the  ownship  and  consort  measured 
bearings  and  the  predicted  bearings  based  on  the 
combined  solution  for  the  case  when  an  invalid  association 
is  made.  In  this  case,  the  combined  solution  is 
significantly  biased  and  the  indicated  quality  is  high  (see 
Fig.  5).  Both  objective  functions  increase  as  expected 
although  the  objective  function  for  ownship  increases  only 
slightly  (from  69.42  to  73.32).  As  expected,  the  value  of 
is  low  (1.0943)  when  the  valid  association  is  made,  and 
is  high  (8.5874)  when  an  invalid  association  is  made. 

From  looking  at  the  residual  plots,  it  is  easy  to  see  how 
difficult  it  is  for  an  operator  to  confidently  make  an 
association  decision,  since  all  the  ’’dot  stacks”  look  quite 
similar.  The  bearing  residuals  will  not  always  get 
significantly  larger  when  an  invalid  association  is  made, 
since  an  algorithm  will  fit  a  viable  target  track  through 
both  sets  of  contact  bearings.  The  true  value  of  the 
operator  will  be  in  evaluating  the  association  based  on 
classification  and  attribute  data  as  well  as  experience. 


INDIVIDUAL  DATA  SETS 


-*(*«) 


VALID  ASSOCIATION 
AND  FUSION 


invalid  association 

AND  FUSION 


Figure  7.  Manual  Association  Residual  Plots 


CORRELATION  OF  TOWED  ARRAY  CONICAL 
ANGLE  HISTORIES  WITH  RADAR  POSITION 
HISTORIES 

Typically,  surface  ships  maintain  a  constant  course  and 
speed  track  for  long  intervals  while  making  passive 
measurements  of  conical  angles  on  many  submarine  or 
surface  contacts.  Without  an  ownship  maneuver,  a 
position  track  cannot  be  derived  from  only  conical  angles. 
Even  if  an  ownship  maneuver  is  performed,  the  position 
of  many  contacts  may  only  be  weakly  observable  (due  to 
poor  geometry)  and  therefore  not  practically  computable. 
If  radar  position  tracks  of  surface  contacts  are  measured 
either  by  ownship  radars  or  radars  on  other  platforms  ,  it 
would  seem  tactically  important  to  attempt  correlation  of 
the  towed  array  conical  angle  histories  with  the  radar 
position  histories.  Successful  correlation  would  allow 
culling  out  of  the  surface  traffic  from  ownship  conical 
angle  contact  files. 

The  correlation  decision  may  be  performed  by  first 
converting  the  radar  position  [A*(/),y(/)]  to  conical  angles, 
fi(t)  as  seen  from  ownship  and  then  a  hypothesis  test  can 
be  performed  on  the  differences  between  these  conical 
angle  histories  as  derived  from  radar  and  the  conical  angle 
histories  as  measured  by  ownship.  To  do  the  jr,y 
conversion  to  conical  angle,  the  Inverse  Raytrace 
algorithm  described  in  Ref  g  could  be  utilized.  The 
correlation  decision  steps  are  as  follows: 

•  Interpolate  x,y  positions  to  times  of  ownship  conical 
angle  positions. 

•  Convert  x,y  positions  to  range,  R,  and  bearing,  B, 
from  ov/nship. 

•  Find  D/E  from  R  using  Inverse  Raytrace  Program 

•  Compute  conical  angle  0  from  cosi?  =  cosBcosD /E 

•  Propagate  confidence  ellipses  (P  matrices)  of  x,y 
positions  into  conical  angle  variances. 

•  Compute  weighted  differences  between  ownship  conical 
angles  and  the  conical  angles  as  derived  above  from 
radar  position  reports.  The  weights  are  derived  from 
the  variances  of  the  ownship  and  the  radar  conical 
angles. 

•  Compare  weighted  differences  to  a  predetermined 
threshold  and  declare  correlated  those  tracks  below  the 
threshold. 


TRACK  SEGMENT  ASSOCIATION 

R.  Mucci  in  Ref  9  describes  an  algorithm  for  deciding 
whether  two  sensors  reporting  track  segments  are 
observing  the  same  target.  The  hypothesis  test  is  ”Is  there 
a  time  for  which  the  two  estimated  trajectories  have  a 
common  position?”  The  test  is  equivalent  to  a  ^  lest  on 
the  weighted  difference  of  the  extrapolated  track  segments 
to  the  common  time;  then  the  track  segments  can  be  re- 
estimated  using  the  combined  set  of  measurements.  The 
technique  allows  for  target  maneuvers.  Figure  9  shows 
two  track  segments  for  a  maneuvering  target  as  reported 
by  two  sensors  and  the  extrapolated  common  time  error 
ellipses.  Figure  10  shows  the  error  ellipse  of  the 
combined  measurements  sets  for  the  common  point  of 
intersection.  This  then  is  a  version  of  the  Temporal  Data 
Association  problem  described  in  Ref  10  by  V.  Gabriel. 
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ELIMINATING  FALSE  TARGETS  FROM  MULTL 
SENSOR  BEARING  DATA 

A  fairly  new  technique  called  simulated  annealing 
(described  in  Ref  11)  can  be  used  to  find  the  lowest 
minimum  of  a  multi-minima  function  and  should  be 
investigated  for  applicability  to  contact  management.  he 
application  to  TMA  is  illustrated  in  I’igure  11  which  shows 
the  multi-minima  cost  function  obtained  by  estimating  two 
target  locations  as  measured  by  bearing  lines  from  three 
sensors.  The  false  target  locations  have  a  higher  minimum 
than  the  true  target  locations;  and  a  conventional  NLLS 
gradient  descent  algorithm  could  be  trapped  at  these  false 
locations. 


STATE  ESTIMATE  HISTORIES 

Although  the  association  algorithm  presen^d  here  uses 
only  two  single  full  state  estimates,  the  x 
modified  to  handle  position-only  state  estimate  histories  as 
demonstrated  in  Reference  1.  Likewise,  full  state 
histories  can  be  used  to  test  whether  2  contacts  are  from 
the  same  source.  Using  histories  of  uncorrelated  position 
estimates  will  result  in  an  improved  association  test,  since 
more  data  will  yield  more  confident  associations. 
However,  if  state  estimates  are  generated  sequentially  by  a 
TMA  algorithm,  like  NLLS,  MPKAST  or  MSKAST,  then 


correlations  between  successive  solutions  will  exist  since 
over-lapping  measurement  sets  are  used  to  generate  the 
solutions.  Therefore,  since  these  estimates  are  correlated, 
using  a  history  of  correlated  estimates  vice  a  single 
uncorretated  estimate  pair  may  not  yield  significantly 
better  association  results.  However,  if  an  algorithm  uses  a 
finite  data  window  (which  NLLS  is  capable  of  doing)  and 
if  the  state  estimates  are  chosen  such  that  there  is  virtua  y 
no  overlap  between  the  measurement  sets  used  in 
generating  the  solutions,  then  the  solution  history 
Wroach  might  yield  significantly  better  data  association 
results  since  there  would  be  virtually  no  correlation  in  the 
state  estimates.  Likewise,  a  short  history  ot  widely  time 
separated  state  estimates  could  yield  better  association 
results. 


CONCLUSIONS 

The  increased  number  of  contacts  resulting  from  sensor 
improvements  and  the  use  of  off-board  data  will  require 
reliable  automatic  multi-sensor,  multi-target  algorithms. 
Solutions  generated  by  these  algorithms  must  be 
automatically  screened  using  a  performance  index.  Also 
reliable  target  maneuver  detection  is  necessary  to  avoid 
incorrect  association  which  can  lead  to  an  extremely 
misleading  tactical  picture.  Manual  performance 
assessment  and  maneuver  detection  is  slow  and  should  be 
avoided  whenever  possible.  Likewise,  manual  geometric 
association  is  difficult  using  the  dot  slack  approach  and 
its  utility  is  questionable.  Simple  statistical  tests,  such  as 
the  central  test,  arc  useful  for  geometric  association, 
however,  these  tests  must  be  accompanied  by  reliable 
attribute  association  schemes,  and  the  association  results 
should  be  periodically  checked  as  more  data  is  received,  in 
order  to  avoid  false  associations.  Since  the  probability  of 
false  associations  can  not  be  directly  specified  and  varies 
greatly  with  both  measurement  quality  and  scenario 
geometry,  and  because  the  TMA  problem  can  be  quite 
troublesome,  false  associations  will  inevitably  occur. 

Geometric  data  fusion  can  be  accomplished  at  both  a 
measurement  level  and  a  track  level.  I  rack  level  lusion  is 
preferred  because  it  provides  more  information,  simplifies 
decorrelation,  provides  for  subsequent  statistical  tests, 
facilitates  the  use  of  an  error  budget,  and  finally  is  natural 
when  processing  measurements  irom  dilferent  sensors.  In 
poorly  observable  scenarios,  track  level  fusion  can  suffer 
numerical  precision  problems  and  the  resulting  biased 
composite  solutions  have  been  demonstrated  even  though 
numerically  stable  orthogonali/ation  techniques  were 
employed  and  double  precision  arithmetic  was  used. 
Under  adverse  conditions,  measurement  level  geometric 
fusion  may  be  the  only  practical  means  of  computing 
reliable  composite  solutions. 
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A  multisensor  approach  to  detection  and  identification  of 
aircraft  is  described.  The  sensors  that  are  utilized  are  a  dual 
mode  long  wavelength  infrared  (IR)  sensor  and  a  imaging  visual 
sensor.  The  IR  sensor  will  be  able  to  perform  in  an  imaging  and 
track  while  scan  mode.  The  detection  is  accomplished  with  the  IR 
sensor  in  a  track-while-scan  mode  (TWS).  The  detection  algorithm 
is  based  on  a  least-mean-square  (LMS)  filter  with  a  post 
screening  algorithm.  Identification  is  performed  using  the 
visual  or  imaging  IR  sensor.  The  aircraft  identification 
algorithm  is  a  knowledge  based  approach  based  on  the  US  Array *s 
"Visual  Aircraft  Recognition  Manual."  The  algorithms  and  system 
design  will  be  described  however  the  emphasis  will  be  on  the 
detection  and  identification  algorithms. 


357 


A  Multisensor  Approach  to  Aircraft  Detection 
and  Identification 


G,  A.  Roberts  and  L.  H.  Bradford 


Ford  Aerospace 
Ford  Road 

Newport  Beach,  CA .  92658 


INTRODUCTION 

Current  aircraft  detection  and 
classification  systems  operate  with  a 
high  degree  of  autonomy.  The  detection 
system  will  typically  just  provide  the 
target  position  to  the  classifier,  A 
synergistic  system  is  presented  which 
consists  of  a  dual  mode  IR  sensor  (IRST 
and  HR),  and  a  visible  imager  in  a 
shared  aperture  and  dual  stabilized 
platform.'  The  detection  is 

accomplished  using  a  LW-IRST  (long 
wavelength  infrared  search  and  track) 
sensor  with  a  detection  processor,  A 
hybrid  detection  algorithm  is  used  with 
a  automatic  sensitivity  control.  The 
classification  is  done  using  imagery 
from  the  visual  imager  (long  range  day 
use)  or  the  IR  imager  (shorter  range 
night  use).  The  classification 
algorithm  is  knowledge  based  which 
utilizes  a  aircraft  parts  analysis 
technique  derived  from  the  US  Army’s 
visual  aircraft  recognition  field 
manual.  The  knowledge  based  system 
stores  three  dimensional  representation 
of  aircrafts  for  matching.  The 
classifier  relies  on  the  detection 
processor  to  provide  target  range, 
velocity,  and  position.  The  knowledge 
based  system  utilizes  this  information 
to  calculate  aspect  for  the  purposes  of 
developing  silhouette  models  from  the 
3-D  aircraft  representations,  and  to 
size  the  models.  The  use  of  aspect  and 
range  greatly  increases  the 
classification  performance.  The 
classifier  is  also  able  to  provide  the 
TWS  processor  with  target/clutter 
information  which  will  enable  it  to 
adjust  the  automatic  detection 
threshold  parameters  to  optimize 
detection  to  false  alarm  performance. 

The  synergism  between  the  detection 
and  classification  greatly  improves  the 
performance  of  both  functions.  The 
knowledge  based  classifier  is  greatly 
assisted  by  being  provided  the  range 
and  velocity  of  the  aircraft.  This 
enables  the  classifier  to  determine 


target  aspect  and  size  more  reliably. 
Without  this  information  the 
classification  performance  will  be 
severly  limited  since  the 
classification  will  need  to  treat  the 
aspect  as  a  degree  of  freedom. 
Additionally  the  classifier  will  be 
able  inform  the  detection  processor 
that  it  is  detecting  clutter.  This 
will  allow  the  detection  processor  to 
adjust  its  detection  sensitivity  so 
that  it  does  not  report  an  excessive 
amount  of  false  detections.  The  basic 
system  will  first  be  described,  then 
the  detection  and  classification 
algorithms  will  be  presented. 


SYSTEM  DESCRIPTION 

The  system  consists  of  a  dual  mode 
IR,  visual  sensor, 
detection/identification  processor,  and 
a  controller.  The  basic  system  is 
shown  in  figure  1.  The  IR  sensor  will 
have  the  capability  of  scanning  a 
narrow  and  wide  field  of  view  in  the 
imaging  mode.  In  the  TWS  mode  it  will 
function  as  an  infrared  search  and 
track  sensor  and  execute  a  gimbal  scan 
while  performing  a  rapid  parallel  scan. 
The  detection/identification  processor 
will  perform  target  detection  using  the 
IR  seeker  and  assemble  a  track  list  of 
the  detections.  Target  range  will  be 
estimated  using  a  passive  ranging 
technique.  When  a  detected  target  is 
in  recognition  range  the  sensor  will 
stare  at  the  target  and  a  visual  sensor 
will  be  used  for  identification.  At 
night  the  IR  sensor  can  be  used  in  an 
imaging  mode  but  the  recognition  range 
will  not  be  as  long  as  a  visual  sensor 
in  clear  sky.  The  difference  is  due  to 
the  greater  resolution  possible  at  the 
visual  wavelengths  as  compared  to  the 
infrared  wavelengths.  In  order  to 
achieve  the  increase  in  resolution  a 
higher  degree  of  stabilization  must  be 
achieved  for  the  visual  sensor. 
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The  detection/identification 
processor  will  consist  of  a  dual  buss 
architecture  with  the  repetitive  high 
computational  operations  such  as  edge 
detection  and  LMS  filtering  being 
performed  by  parallel  processors  off  a 
high  speed  image  buss.  The  lower 
computational  and  more  hierarchical 
operations  will  be  executed  by  using 
programmable  processor  modules  off  a 
VME  buss.  The  demonstration  processor 
utilizes  Datacube  processor  cards  in 
addition  to  specially  built  processors 
off  a  high  speed  image  buss.  Motorola 
68020  processors  are  also  used  for  the 
lower  computational  computations.  This 
configuration  allows  flexibility  of 
programing  the  higher  level  operations 
while  having  expandable  lower  level 
primitives . 


AIRCRAFT  DETECTION 

Aircraft  detection  will  be  done  by 
using  the  imagery  from  the  IR  sensor 
while  it  is  in  the  TWS  mode.  The 
detection  algorithm  structure  is  shown 
in  figure  2.  The  algorithm  uses  a  LMS 
filter  as  a  preliminary  screening 
algorithm  [1,2]  to  identify  areas  of 
the  image  that  may  contain  targets. 
The  filter  responses  are  automatically 
thresholded  using  a  polynomial 
expansion  technique  to  extrapolate  the 
proper  threshold  [31.  The  threshold 
exceedances  are  then  screened  further 
using  a  connectivity  C3]  process  with 
several  gate  sizes.  The  connectivity 
algorithm  has  better  performance  than 
the  LMS  algorithm  but  is  more  difficult 
to  implement  for  the  entire  image. 
Because  of  this,  the  LMS  is  used  as  a 
preliminary  screening  algorithm.  The 
connectivity  algorithm  basically  tests 
the  detection  to  see  if  it  is  connected 
to  other  objects  in  the  scene.  The 
detection  must  be  isolated  in  order  to 
consider  the  object  as  a  target.  The 
detections  that  pass  the  connectivity 
screening  are  assembled  into  a  track 
file  utilizing  frame-to-frame 
correlation  of  spacial  location  and 
intensity  in  the  image.  The  track 
analysis  module  will  analyze  the  track 
for  merit  and  prioritization. 

In  some  cases  the  sensor  must  look 
down  to  detect  an  aircraft.  In  this 
circumstance,  there  may  be  high  clutter 
in  the  scene.  The  automatic 
thresholding  module  will  sense  the 
clutter  content  and  inform  the 
detection  algorithm  of  this  condition 
and  a  high  clutter  analysis  will  be 
done.  In  high  clutter  the  target 
cannot  be  uniquely  separated  from  the 
other  target-like  objects  in  the  scene 
since  they  will  have  the  same 
target-like  signatures  as  the  actual 
aircraft.  When  activated  the  high 
clutter  module  will  lower  the  automatic 
threshold  enabling  many  false  targets 
with  target-like  signatures  to  pass  the 
LMS  and  connectivity  screening.  The 
high  clutter  module  will  then  analyze 
the  detections  for  motion,  local  image 
context,  and  shape.  If  a  potential 


target  has  sufficient  evidence  to 
identify  it  as  an  aircraft  then  it  will 
be  reported  as  a  detection.  An  example 
of  aircraft  detection  in  a  high  clutter 


A  passive  range  estimate  will  also 
be  performed  using  the  angular  motion 
of  the  potential  target.  For  a  moving 
platform  this  will  be  accomplished  by 
having  the  platform  execute  a  maneuver. 
The  relative  motion  of  the  potential 
target  before  is  compared  with  the 
relative  motion  after,  and  the  target 
range  is  calculated  using  the  change  in 
velocity  of  the  platform  and  assuming 
the  target  velocity  did  not  change. 
For  stationary  platforms  the  rate  of 
growth  of  the  target  coupled  with  the 
angular  velocity  of  the  target  will 
determine  the  velocity  direction. 
Knowing  the  direction,  then  a  range 
assumption  can  be  calculated  for  each 
class  of  aircraft  by  comparing  the  the 
size  of  the  target  to  the 
characteristic  dimensions  of  the 
aircraft  at  that  aspect, 

AIRCRAFT  IDENTIFICATION 

Aircraft  identification  is 
accomplished  by  using  imagery  from  the 
visual  sensor  during  the  day  and  from 
the  IR  sensor  at  night.  When  the  size 
of  the  detected  aircraft  becomes  large 
enough  to  be  identified,  the  sensor 
will  stare  at  the  aircraft  and  an 
identification  will  be  attempted.  In 
order  to  identify  an  aircraft,  the 
aspect  and  range  are  useful.  This  data 
enables  a  model  based  approach  to  be 
used  which  improves  classification 
performance.  The  basic  approach  us3d 
for  aircraft  identification  was 
motivated  by  the  US  Army  "Visual 
Aircraft  Recognition  Manual"  [4],  This 
manual  serves  as  an  Army  training 
manual  for  aircraft  recognition.  The 
manual  indicates  that  humans  identify 
aircraft  by  looking  for  distinguishing 
characteristics  of  aircraft  parts  such 
as  distinctive  shapes  or  configurations 
of  wings,  fuselages,  vertical 
stabilizers,  or  other  aircraft  parts. 
Based  on  this  information,  a  knowledge 
based  system  was  designed  which 
utilized  a  parts  analysis,  A  model 
generating  module  was  developed  for 
this  system  to  provide  references  for 
the  aircraft  parts.  This  model 
generator  can  generate  a  crude 
silhouette  for  any  of  the  aircraft 
classes  for  any  aspect  angle.  The 
parts  of  the  aircraft  models  can  then 
be  matched  to  the  aircraft  image  using 
a  parts  matching  technique.  An 
aircraft  class  is  declared  based  on  the 
parts  analysis.  The  knowledge  based 
system  consists  of  several  modules. 
These  modules  are  the  model  guided 
segmentation,  aspect  determination, 
model  generator,  and  parts  matching. 
The  model  guided  segmentat-on  module 
segments  the  aircraft  based  on  a  crude 
background  model.  The  aspect 
determination  module  determines  the 
aircraft  pitch,  yaw,  and  rol3  angles  by 
using  motion  or  a  skeleton  approach. 
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The  parts  matching  module  will  match 
model  parts  to  the  aircraft  parts.  The 
model  generator  generates  models  of  the 
aircrafts  for  the  determined  aspect. 
An  outline  of  this  system  is  shown  in 
figure  4, 

The  segmentation  procedure  uses  a 
region  segmentation  called  the 
Histogram  Optimization  Segmentation 
(HOS)  C5,6]  (this  is  similar  to  edge 
preserving  smoothing  [7,8])  followed  by 
a  model  guided  merging  [9],  The  HOS 
algorithm  applies  an  adaptive  3x3 
filter  which  smooths  areas  of  uniform 
grey  level  while  enhancing  edges.  The 
filter  is  applied  until  it  converges. 
At  this  stage  a  simple  form  of 
clustering  which  groups  adjacent  pixels 
that  have  an  intensity  difference  below 
a  segmentation  threshold  [5,6]  is 
applied.  This  grouping  creates  the 
image  segments.  The  segmentation 
threshold  must  first  be  determined  to 
accomplish  the  segmentation.  This  is 
done  automatically  by  resampling  the 
image  so  that  the  anticipated  target  is 
only  a  few  pixels  in  the  image.  Then 
the  magnitude  of  the  contrast  of  the 
adjacent  points  in  the  resampled  image 
is  determined  and  a  contrast  histogram 
is  created.  Based  on  the  contrast 
histogram  two  trial  segmentation 
thresholds  are  examined.  Each  one  of 
these  segmentations  is  identified  with 
a  hypothesis.  One  hypothesis  assumes 
that  the  target  has  the  largest 
contrast.  Another  assumes  that  the 
target  contrast  is  essentially  the 
contrast  of  the  background  objects. 
Each  one  of  these  trial  segmentation 
thresholds  is  executed  then  a 
hypothesis  verification  is  done.  The 
verification  is  done  by  examining  the 
size,  spacial  distribution,  and  number 
of  segments  in  the  trial  segmentations. 
Characteristics  such  as  these  are  used 
to  confirm  a  successful  trial 
segmentation.  For  example  a  high 
confidence  is  associated  with  the  first 
trial  segmentation  if  only  one  small 
segment  is  present  after  segmentation. 
Also  a  low  confidence  is  associated 
with  the  first  segmentation  if  many 
small  segments  are  present  in  the  trial 
segmentation.  Similarly  a  high 
confidence  is  associated  with  the 
second  trial  segmentation  if  it  has 
only  one  small  segment. 

After  the  HOS  segmentation  is 
accomplished,  then  a  merging  algorithm 
is  used.  The  algorithm  uses  rules  that 
are  aimed  at  combining  segments  that 
are  large  and  uniform  throughout  the 
image.  This  is  done  since  the  sky 
background  is  modeled  as  a  large  area 
of  relatively  uniform  contrast  changes. 
The  merging  algorithm  merges  the 
background  pieces  then  labels  the 
remaining  segments  as  a  target. 

One  advantage  of  designing  a  system 
to  recognize  aircraft  is  that  once  a 
target  velocity  vector  is  known  then 
the  yaw  and  pitch  of  the  aircraft  is 
determined.  This  is  due  to  the 
observation  that  in  normal  flight,  an 


aircraft  will  fly  with  their  major  axis 
(the  axis  between  the  nose  and  tail) 
parallel  to  their  velocity  vector. 
Additionally,  once  their  pitch  and  yaw 
are  known  then  simple  features  (  such 
as  the  width  to  length  ratio  and  wing 
skeleton  angles)  can  be  used  to 
identify  the  roll  angle.  The  method 
used  to  determine  the  velocity  vector 
is  to  measure  the  angular  displacement 
of  the  aircraft  in  the  image  along  with 
the  change  in  length  of  the  aircraft. 
These  parameters  are  sufficient  to 
determine  the  direction  of  the  aircraft 
velocity  vector  for  a  stationary 
observer.  For  the  moving  observer  at 
least  the  observer’s  velocity  and  the 
range  to  the  aircraft  need  to  be 
estimated  or  measured.  To  show  that 
the  velocity  direction  can  be  found  for 
the  aircraft  using  image  motion, 
consider  that  the  aircraft  velocity  is 
given  by  the  following  equation: 


V  =  dR 
dt 
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where  r,  0,  and  <J>  are  the  sensor 
pointing  angles  to  the  aircraft  and 
seen  in  figure  5.  Assuming  that  the 
velocity  is  uniform  over  a  time  At  then 
the  change  in  the  aircraft  length  in 
the  image  (AL)  is  given  by:  8  " 
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where  L  is  the  original  length 
is  the  change  in  range, 
above  in  equation  (1)  yields: 
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velocity  vector.  The  angles  0  and  <|> 
can  be  computed  by  finding  the  change 
in  location  of  the  aircraft  through 
images  separated  by  time.  The  change 
in  length  of  the  aircraft  can  also  be 
computed  through  time  by  computing  the 
change  in  the  major  axis  length  of  the 
target  segment.  Figure  6  shows  a 
sequence  of  Images  where  the  direction 
of  the  velocity  vector  is  computed  in 
this  manner. 

In  addition  to  using  motion  to 
determine  aspect,  the  use  of  a  type  of 
skeleton  [10,11]  matching  has  been 
investigated  for  this  problem.  This 
technique  uses  a  pseudo-skeletal 
analysis  which  differs  from  the 
conventional  medial  axis  transform. 
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The  aircraft  is  decomposed  into  a 
skeleton  then  the  skeleton  is  matched 
to  skeletons  of  the  models  at  different 
aspects.  The  aspect  of  the  best 
matching  model  skeleton  is  used  as  the 
aircraft  aspect.  This  aspect 
determination  is  done  for  each  class  of 
objects.  The  method  used  is  to  limit 
the  angle  search  to  two  angles.  The 
approach  used  to  generate  the 
pseudo-skeleton  is  to  initially  find 
the  longest  axis  that  can  fit  in  the 
segmented  aircraft.  Then  appendages 
that  extend  from  this  axis  are  found 
(see  figure  7).  The  major  axis  is 
derotated  to  horizontal  (this 
derotation  angle  is  either  the  pitch  or 
roll  of  the  aircraft  depending  on  the 
matching  skeleton).  The  derotated 
skeleton  is  then  matched  to  a  set  of 
stored  skeletons  for  a  particular  class 
to  find  the  yaw  and  roll  or  the  yaw  and 
pitch.  The  matching  is  accomplished 
using  cost  functions  [12], 

A  model  based  (top-down)  approach 
[133  is  used  to  match  the  model 
aircraft  parts  to  the  segmented 
aircraft.  The  classifier  carries  the 
capability  to  generate  a  reference 
silhouette  for  a  model  for  any  aspect. 
The  models  are  stored  as  a  set  of 
3-dimensional  points.  Each  point  is 
the  corner  of  a  planar  surface.  A 
complex  representation  was  not  adopted 
since  simple  models  seem  to  be 
effective  and  require  fewer 
computations  to  generate  [93,  The 
surfaces  were  determined  by  examining 
aircraft  outlines  contained  in 
Reference  4,  The  vertex  points  were 
transformed  according  to  the  pitch, 
yaw,  and  roll  angles  which  are 
determined  from  the  aspect 
determination  module.  An  example  of 
the  model  generation  for  various  aspect 
is  shown  in  figure  8.  Models  for  the 
A-Y,  F/A-18,  F-5,  F-111,  MiG-25,  B-52, 
Tu-26,  F-15,  and  MiG-21  aircraft  are 
stored.  Additionally  each  surface  is 
labeled  according  to  its  aircraft  part. 


its  visible  parts  match  with  good 
confidence.  Allowances  are  made  for 
part  matches  where  the  part  is  small 
and  therefore  can  be  easily  . ost  during 
segmentation  or  imaging, 

IDENTIFICATION  COMPARISON  REbdLTS 

Two  statistical  classifiers  were 
compared  to  the  knowledge  based 
classifier  in  order  to  determine  if  it 
was  more  accurate  than  conventional 
classifiers.  These  statistical 
classifiers  were  near  neighbor 
classifiers  that  used  the  sane 
segmentation,  aspect  determination,  and 
model  generation  as  the  knowledge  based 
classifier.  However  the  statistical 
classifier  would  classify  based  on 
features  such  as  Hu*s  moments  or 
discrete  fourier  transforms  of  the 
boundary.  These  features  were  stored 
as  a  function  of  aspect.  The  features 
were  chosen  based  on  maximizing  the 
F-statistic  through  the  training  data. 
In  this  analysis  two  groups  of  features 
were  found  that  had  essentially 
equivalent  F-statistics  so  two 
classifiers  were  used  with  each  One 
using  one  of  the  feature  groups.  The 
classifiers  were  compared  using  real 
imagery  and  using  a  controlled  test. 
When  real  imagery  was  used  the 
knowledge  based  classifier  performed 
much  better  that  the  statistical 
classifier,  however  the  performance 
difference  was  a  strong  function  of  the 
test  data  so  a  more  controlled  test  was 
needed.  A  controlled  test  was 
accomplished  where  noise  of  varying 
amplitude  was  added  to  aircraft 
silhouettes  then  the  degraded 
silhouettes  were  segmented  and  run 
through  the  classifiers.  An  example  of 
adding  the  noise  is  shown  in  figure  9. 
The  results  of  this  comparison  are 
shown  in  figure  10,  It  can  be  seen 
that,  for  the  four  aspects,  the 
knowledge  based  classifier  performed 
significantly  better  than  the  two 
statistical  classifiers. 


A  parts  matching  is  used  to 
classify  the  aircraft.  This  parts 
matching  is  done  in  a  top  down  manner 
based  on  the  identified  parts  of  the 
class  models.  A  match  is  done  on  the 
nose,  body,  wings,  vertical 

stabilizer(s),  and  horizontal 

stabilizers.  A  present  time  a  simple 
area  match  confidence  is  used.  The 
technique  is  to  size  the  generated 
model  so  that  its  area  is  equal  to  the 
segmented  aircraft,  then  align  their 
centroids.  A  confidence  of  match  is 
determined  for  each  part  of  the  model 
by  computing  the  ratio  of  the 
intersection  of  the  model  part  and  the 
segmented  aircraft  to  the  area  of 
model  part.  In  this  manner 
confidence  measure  is  determined 
each  of  the  different  model^s  parts 
simple  rule  system  then  makes  a 
classification  decision  based  on  the 
confidence  measures  and  the  size  of  the 
model  parts.  The  system  declares 
aircraft  to  be  a .  particular  class 
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Additional  tests  also  performed  to 
analyze  the  performance  of  the 
knowledge  based  classifier  to  the 
statistical  classifiers  for  different 
range  targets.  In  this  test  targets 
were  simulated  for  different  ranges  by 
undersampling  the  resolution  (see 
figure  11).  Other  effects  such  as 
atmospheric  effects  were  not  into  the 
model  so  as  to  isolate  the  effect  of 
special  sampling  on  the  different 
classifiers.  The  comparative  results 
for  different  ranges  are  shown  in 
figure  12.  It  can  be  seen  that  the 
knowledge  based  classifier  performed 
significantly  better  than  the 
statistical  classifiers. 

CONCLUSION 

The  synergistic  approach  to  using 
the  information  from  both  the  detection 
and  identification  systems  will 
increase'  the  performance  of  both 
systems.  The  identification  is  greatly 
assisted  by  being  provided  a  passive 
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range  and  velocity  vector  of  the 
aircraft.  This  allows  a  model  based 
approach  to  be  used  which  as  this  study 
shows  has  a  much  better  performance 
over  the  conventional  classifiers.  The 
classifier  can  also  assist  detection  by 
providing  target  and  clutter 
information  which  can  help  the 
detection  process  determine  the  optimal 
detection  threshold. 
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Figure  1;  The  detection  and  identification  system  using  a 
IIR/IRST  sensor  and  a  visual  sensor. 
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Figure  3;  The  detection  without  false  alarms  of  the  moving 
aircraft  (boxed  target  in  center  of  image)  in  high  clutter. 
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•  PLATFORM 
VELOCITY 

IMAGE  •  target 


Figure  4;  Outline  of  the  knowledge  based  system  for  aircraft 
recognition , 


Figure  5;  The  pointing  angles  to  an  aircraft  (  0  and  <|>  )  and 
the  range  (R). 
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IMAGE  1 


image  2 


A0  =  2  X 
A0  =  10”^ 

2  X  10“2 


0  =  -.2? +.20-  .9<^ 


Figure  6;  The  velocity  vector  computed  from  interframe  motion. 
The  image  displacement  and  change  in  length  of  the  aircraft  is 
used  to  determine  the  direction  of  the  velocity  vector. 
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PITCH  =  0 
YAW  =  1 .47 
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MATCHING  MODEL 
SKELETON 

Figure  7;  The  skeletal  matching  procedure.  An  image  is 
segmented  then  the  major  axis  and  the  skeletons  is  derotated 
and  matched  to  models  thus  finding  the  roll  angle. 


Figure  8;  Various  aspects  of  a  Tu-26  (Backfire)  bomber 
generated  with  the  model  generator. 
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Figure  9;  An  example  of  degrading  a  target  silhouette  with 
noise. 
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KNOWLEDGE  BASED  CLASSIFIER,  O  = 
STATISTICAL  CLASSIFIER  2,  Pc  = 


PROBABILITY  OF  CORRECT  CLASSIFICATION 


Figure  10;The  comparative  probability  of  correct 
classification  for  the  knowledge  based  classifier  and  the 
statistical  classifier  as  a  function  of  signal-to-noise  ratio 
(SNR) . 
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RANGE=R1  RANGE=:2R1  RANGE=4R1 


Figure  11;  Examples  of  the  spacial  sampling  degrading  of  a 
aircraft  at  various  ranges. 


A  =  KNOWLEDGE  BASED  CLASSIFIER,  O  =  STATISTICAL  CLASSIFIER  1. 


□  =  STATISTICAL  CLASSIFIER  2,  ^  PROBABILITY  OF  CORRECT  CLASSIFICATION 


Figure  12;The  comparative  probability  of  correct 
classification  for  the  knowledge  based  classifier  and  the 
statistical  classifier  as  a  function  of  spacial  degradation 
range. 
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USE  OF  A  TRACK-TO-TRACK  ASSOCIATION  ALGORITHM 
IN  THE  FUSION  OF  LONG-RANGE  SENSOR  DATA 
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ABSTRACT 

A  computer-based  algorithm  is 
described  that  performs  the  fusion  of 
ship  tracks  from  different  sensors  or 
data  sources  but  which  pertain  to  the 
same  platform.  It  is  applicable  to 
long-range  sensor  data  for  the 
Over-the-Horizon  Targeting  (OTH-T) 
function.  As  a  component  of  a  Tactical 
Data  Processor  (TDP) ,  the  algorithm 
conducts  pairwise  comparison  of  tracks 
on  the  basis  of  positional  statistical 
distance.  Initial  screening,  based  on 
some  attributes,  age  of  data,  and 
position  feasibility,  filters  out 
candidate  tracks  that  are  clearly 
unrelated  or  of  no  interest. 

Evaluation  of  the  algorithm  to  date 
shows  an  advantage  over  manual  fusion 
both  in  process  time  and  capability  to 
find  track  merge  candidates. 


INTRODUCTION 

Over-the-Horizon  Targeting  (OTH-T) 
requires  the  correlation  and  fusion  of 
long-range  sensor  data  from  a  number  of 
different  sources.  Because  of  the 
characteristics  of  these  sensors  and 
sources,  it  frequently  happens  that  a 
given  ship  platform  is  represented  by 
two  or  more  constructed  tracks 
corresponding  to  different  sensors  or 
sources . 

In  order  to  produce  an  accurate 
OTH-T  picture,  it  is  necessary  to 
identify  and  merge  these  tracks 
together.  This  function  is  usually 
referred  to  as  track-to-track 
association.  Currently,  there  is  no 
automatic  track-to-track  association 
function  in  any  of  the  existing  OTH 
systems.  With  the  development  of  new 
sensors  and  sources  of  OTH-T  data,  such 
a  function  is  increasingly  important. 


Tiburon  Systems  has  developed  an 
experimental  track-to-track  association 
algorithm,  based  on  a  Kalman-filter 
tracking  algorithm  called  the 
Maneuvering  Target  Statistical  Tracker 
(MTST) .  The  initial  development  work 
was  performed  as  part  of  an  independent 
research  and  development  project  within 
Tiburon  (Ref.  1) .  Additional  work  in 
this  area  has  been  funded  by  the  Naval 
Underwater  Systems  Center  (NUSC) , 

Cruise  Missiles  Project  Office  (CMPO) , 
and  the  Space  and  Naval  Warfare  Systems 
Command  (SPAWAR) . 

The  algorithm  performs  pairwise 
comparisons  among  tracks  in  a  data 
base,  and  identifies  tracks  that  are 
recommended  to  be  merged.  The 
algorithm  includes  initial  screens  to 
detect  tracks  that  are  incompatible  on 
the  basis  of  their  attributes,  age 
difference,  and  positions.  A  track 
pair  that  passes  the  initial  screens  is 
subjected  to  a  detailed  positional 
analysis  which  results  in  a  decision 
whether  to  reject  the  track  pair  as 
merge  candidates,  to  include  the  track 
pair  as  merge  candidates,  or  to  defer 
the  decision  until  more  data  are 
available . 


BASIS  OF  THE  MEASURE  OF  ASSOCIATION 

The  measure  of  association  is 
based  on  the  average  of  the  statistical 
distances  between  contacts  of  a 
comparison  track  to  the  filtered, 
smoothed  solutions  of  the  reference 
track  at  the  times-of-event  (TOEs)  of 
the  contacts.  This  approach  is  an 
extension  of  the  concept  of  testing  for 
an  outlier  contact  in  a  single  track, 
as  described  in  Reference  2  regarding 
contact  reliability.  In  this  approach, 
the  contacts  (excluding  the  contact 
being  tested)  are  used  to  forward 
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filter  and  backward  smooth  the  track. 
The  statistical  distance  is  computed 
between  the  contact  arid  the  smoothed 
solution  at  the  contact  time-of-event . 
This,  is  the  "jackknifed"  solution.  The 
measure  has  an  M  degrees-of-freedom 
Chi-square  distribution,  where  M  is  the 
dimension  of  the  position  vector  (M  =  2 
for  position  only,  and  4  for  a  full 
state  report) .  If  the  test  statistic 
has  a  small  probability  of  occurrence, 
the  contact  is  deemed  an  outlier.  The 
test  has  been  shown  in  Reference  3  to 
be  equivalent  to  a  likelihood  ratio 
test  of  the  null  hypothesis,  that  the 
contact  is  a  member  of  the  track, 
against  the  alternative  hypothesis, 
that  the  contact  is  displaced  from  the 
smoothed  track.  This  is  the  "slippage" 
model . 

In  the  track-to-track  association 
algorithm,  the  contacts  of  one  track 
are  tested  individually  against  the 
smoothed  "reference"  track.  This  may 
be  viewed  as  a  multi-dimensional 
"jackknifed"  or  a  "slippage"  model.  The 
quadratic  statistical  distance  is 
defined  as  follows: 


T(k)  =  [Y(k)-  ^(k|n)]'*  ^ 

[S  (k,k|n)+R(k)  *  [Y  (k)-^i(kln)  ] 


where 

Y(k) 

is  the  position  of  the  kth 
candidate  contact. 

|i(kln) 

is  the  smoothed  solution  of  the 
reference  track  (having  n  contacts) 
at  the  TOE  of  Y (k)  , 

R(k) 

is  the  position  covariance  of  the 
kth  candidate  contact,  and 

£(k,k|n) 

is  the  position  covariance  of 
the  smoothed  solution  of  the 
reference  track  at  k. 

In  the  track-to-track  measure,  N 
such  statistical  distances  are 
computed,  one  for  each  contact  of  the 
candidate  track  over  a  sequence  of 
contacts  {JK}  that  lie  within  the  time 
span  of  the  reference  track  when  time 
overlap  is  present,  or  the  end  contacts 
when  the  tracks  are  non-overlapping  in 
time.  The  measure, TAU,  is  the  distance, 
T(k),  averaged  over  k=  JK (1) , . . . , JK (N) . 

If  the  contact  is  a  member  of  the 
track,  which  is  the  null  hypothesis, 
the  distribution  of  TAU  will  be 
Chi-square  with  M  degrees  of  freedom 
for  N  =  1;  for  N  >  1,  the  distribution 
must  be  found  empirically  because  the  N 
measures  are  not  independent. 


pertaining  to  the  same  platform. 

However,  the  test  cannot  effectively 
discriminate  two  tracks  of  platforms 
traveling  in  "parallel"  where  the 
separation  distance  is  in  the  same 
order  of  magnitude  as  the  areas  of 
uncertainty  (AOU)  of  the  sensor 
measurements.  A  conservative  approach 
to  thresholding  for  accepting  the  track 
pair  for  merging,  postulates  an 
alternate  hypothesis  that  assumes  the 
contacts  to  be  from  a  track  that  is 
"parallel"  to  the  reference  track  at  a 
distance  d.  This  simulates  a  "worst 
case"  alternative  hypothesis.  In  this 
case,  for  N  =  1,  the  distribution  will 
be  a  non-central  chi-square  with  a 
non-centrality  parameter  based  on  the 
distance  d.  For  N  >  1,  the 
distribution  will  be  empirical  based  on 
a  non-centrality  parameter  defined  as 
follows : 

X,  =  (  ) 

averaged  over  k  =  JK(1),  ...,  JK{N) 


where 

d  is  the  minimum  separation  distance 

between  parallel  paths  that  is  to  be 
discriminated,  and 

dk  is  the  standard  deviation  of  the 
joint  distribution  of  position 
uncertainty  of  the  two  tracks  at  the 
TOE  of  the  kth  contact,  i.  e.. 


ak^  =  (A  +  V  A*A  -  4*B)/2 


where 

A  =  trace  of  (  R(k)  +  X(k,k|n)  ) 

B  =  determinant  of  (  R{k)  +  Z(k,k|n)  ) 

The  probability  densit  function  of  TAU 
is  approximated  by  the  gamma 
distribution 

gi  (X)  =  -  [ad)  e'  * 

r[r(i)  ] 

where  i  =  0, 1  pertains  to  the  null  and 

alternate  hypotheses,  respectively 

r[r(i)]  is  the  gamma  function  of  r(i) 

ad)  =  |id)  /  o(i)^ 

rd)  =  ^d)^  /  ad)^ 


Based  on  the  statistical 
distribution  of  TAU,  a  threshold  may  be 
established  to  reject  a  track  pair  as 


and  the  parameters  |i{i)  and  a(i)  are  as 
follows : 


Parameter 

(i=0)  NULL  HYPOTHESIS 

(i=l)  ALTERNATE 

N  >  1 

HYPOTHESIS 

N  =  1 

p(i) 

(2.0  +  Cl) 

pom  +  Plm  ^ 

1  +  X/2 

0(1) 

(2.0  +  Cl)  )  /Vn 

(Pos  +  pis  A.)/Vn 

*n/i  +  X. 
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where  Cl  is  the  complexity  of  the 
reference  track.  Complexity  is  defined 
as  the  ratio  of  the  distance  from  the 
first  to  the  last  contact  on  the  track 
to  the  sum  of  the  distances  between 
adjacent  contacts.  If  the  ratio  is 
less  than  0.7,  the  track  is  complex  and 
Cl  =  1;  otherwise,  the  track  is  simple 
and  Cl  =  0.  The  coefficients,  p,  are 
derived  experimentally  and  are  a 
function  of  the  distance  parameter,  d, 
(in  nm)  and  the  complexity  Cl,  as  is 
shown  in  Table  1. 


that  have  new  contacts  added  or 
deleted,  or  that  have  been  merged  with 
another  track.  The  Correlator  Tracker 
processes  all  constructed  tracks  using 
the  Maneuvering  Target  Statistical 
Tracker  (MTST)  of  Reference  2.  The 
purpose  of  this  filtering  is  to  remove 
outlier  (aberrant)  contacts  from  the 
track,  and  to  provide  a  smoothed  track 
based  on  the  "active"  contacts  that 
passed  the  outlier  testing.  The 
original  contact  data,  the  smoothed 
track  data,  and  the  variables  that 


TABLE  1 

VALUES  OF  EMPIRICAL  COEFFICIENTS 


COMPLEXITY 


C1  =  0 
(SIMPLE) 


Cl=l 

(COMPLEX) 


SYMBOL 

d<l 

l^d^3 

om 

1.82 

1.81 

Im 

0.00 

1.75 

os 

2.30 

2.44 

Is 

0.00 

1.73 

om 

2.95 

2.71 

Im 

0.00 

2.05 

os 

4.36 

4.52 

Is 

0.00 

1.03 

3<d<5 

5<cl510 

10<d 

1.80 

1.86 

2.31 

1.70 

1.65 

1.61 

2.76 

3.79 

5.48 

1.48 

1.36 

1.92 

2.36 

1.08 

-1.62 

2.00 

1.95 

1.94 

4.07 

2.88 

0.46 

1.42 

1.61 

1.65 

The  M-score  thresholds  are  based 
on  the  type  I  and  II  errors  of 
hypothesis  testing:  type  I  is  the 
probability  of  rejecting  the  null 
hypothesis  when  it  is  true,  and  type  II 
is  the  probability  of. accepting  the 
alternate  hypothesis  when  it  is  false. 
The  thresholds  are  based  on  the 
incomplete  gamma  functions  G(x;  r,  a): 

The  R-score  is  the  probability  of 
making  a  type  I  error: 

R-SCORE  =  1  -  Go  (TAU;  r(o),  a(o)) 

The  M-score  is  the  probability  of 
making  a  type  II  error: 

M-SCORE  =  1  -  Gi  (TAU;  r(l),  a(l)) 

The  thresholds  of  the  R-score  and 
M-score  are  Ro  and  Mo,  respectively. 
They  are  input  control  parameters.  The 
thresholding  is  as  follows:  If  R-score 
<  Ro,  the  candidate  track  is  rejected 
as  belonging  to  the  reference  track. 

If  M-score  =  Mo,  the  candidate  is 
recommended  for  merging  with  the 
reference  track.  Otherwise,  the 
candidate  is  retained  for  further 
testing. 


DESCRIPTION  OF  ALGORITHM 

The  Track  Association  Algorithm 
(TAA)  is  configured  to  be  a  component 
of  a  Tactical  Data  Processor  (TDP) . 
Figure  1  shows  a  system  in  which  the 
identified  blocks  are  functions 
performed  by  the  TDP . 

On  the  input  side,  the  TAA  Input 
Control  initiates  a  cold  start/warm 
start,  and  manages  the  Track  Update 
Queue  to  the  TAA.  Track  updates  may 
consist  of  tracks  that  are  deleted. 


enable  the  smoothed  track  to  be 
interpolated  and  extrapolated,  are 
stored  in  the  Tracker  Support  File.  In 
conjuction  with  the  TDP  Data  Base 
Management  System  (DBMS) ,  the  track 
attributes  are  stored  in  the  Attribute 
Data  File.  The  TAA  Control  Parameters 
Man  Machine  Interface  (MMI)  permits  the 
adjustment  of  a  set  of  control 
parameters. 

On  the  output  side,  the  TAA  produces 
Merge  Recommendation  Data,  a  Candidate 
List,  and  an  Operator  Reject  List.  The 
TDP  utilizes  these  data  to  make  track 
merge  decisions  and  resolve  conflicting 
merge  recommendations.  A  typical 
conflict  exists  when  track  A  is 
recommended  for  merging  with  tracks  B 
and  C,  but  where  tracks  B  and  C  have 
been  rejected  as  the  same  platform. 

The  TAA  Operator  Query  may  request  that 
a  Console  Selected  track  (CST)  be 
evaluated  versus  all  candidates,  and 
the  candidates  ranked  in  order  of  their 
M-scores.  An  ELINT  Support  Data  Base 
may  be  used  to  help  resolve  merge 
recommendation  conflicts,  or  evaluate 
choices  among  multiple  merges,  by 
correlating  ELINT  parameters  with 
non-ELINT  attributes  at  a  level  higher 
than  that  performed  in  the  attribute 
screening.  The  Merge  Processing  Man 
Machine  Interface  (MMI)  is  the  function 
used  by  the  operator  to  decide  which 
tracks  to  merge;  a  decision  to  merge 
two  tracks  is  conveyed  to  the  Data  Base 
Management  System  (DBMS)  which  will 
perform  the  track  merging  and  update 
the  Track  Update  Queue.  The  Merge 
Processing  MMI  may  also  update  the 
Operator  Reject  List  (as  a  result  of 
conflict  resolution  and  merge 
decisions)  which  causes  the  TAA  to 
exclude  the  track  in  question  from  the 
candidate  list  and  future  merging.  A 
Status  Query  option  is  available  so 
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Figure  1 


that  the  TDP  may  examine  the  status  of 
the  Candidate  List  and  Operator  Reject 
List. 


Figure  2  shows  the  block  diagram 
of  the  TAA.  The  four  major  functional 
modes  are  shown.  The  Compare  mode  is 
the  principal  mode.  The  algorithm 
automatically  compares  each  track  in  a 
source  list  to  all  tracks  in  a 
candidate  list.  Initially,  the 
candidate  list  is  determined  by  the 
source  (s)  of  the  contact  data  in  the 
track.  After  the  first  pass,  the 
candidate  lists  contain  only  candidates 
that  have  not  been  rejected. 
Subsequently,  however,  a  rejected  track 
may  be  reinstated  in  the  candidate  list 
if  the  TDP  has  determined  that  the 
track  has  changed.  A  change  occurs  if 
any  of  the  specified  MTST  parameters 
has  changed,  if  any  active  contact  from 
the  earlier  track  is  deleted  or  becomes 
inactive  (except  the  eldest  contact) , 
or  if  there  is  a  position  change  in  one 
of  the  previously  active  contacts. 


For  an  initial  pass  of  any  track,  the 
Screen  Candidate  function  is  performed. 

A  candidate  may  be  rejected  at  this 
level  on  the  basis  of  attribute 
inconsistencies  and  position 
infeasibility.  Candidates,  whose  age 
difference  with  respect  to  the 
reference  track  is  outside  a  region  of 
interest,  are  retained  in  the  candidate 
list  but  are  not  further  analyzed. 

The  measure  of  association  is 
evaluated  for  candidates  which  pass  the 
screening  tests,  and  provided  that  at 
least  one  of  the  tracks  in  the  pair 
have  multiple  contacts .  The  measure  is 


computed  for  one  track  as  the  reference 
track,  and  the  candidate  as  the 
comparison  track,  and  then  with  the  two 
tracks  in  reverse  roles.  An  exception 
occurs  when  either  track  contains  only 


Figure  2 
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a  single  contact.  In  this  case,  only 
the  multi-contact  track  may  serve  as  a 
reference  track  because  the  expansion 
of  a  single  contact  is  judged  to  be 
unreliable.  If  two  R-scores  and 
M-scores  are  obtained,  they  are  not 
necessarily  symmetric. 

The  logic  for  making  the  decision  to 
recommend  a  merge  is  as  follows :  The 
candidate  is  rejected  if  one  of  the  R-scores 
has  failed  to  pass  the  reject  threshold.  If 
otherwise,  the  candidate  is  recommended  for 
merge  if  one  of  the  M-scores  passes  the 
merge  threshold  and  provided  that  both 
tracks  are  multi-contact.  Otherwise,  the 
candidate  is  placed  on  hold. 

Merge  recommendations  are  recorded  in 
the  Merge  Recommendation  Data  file.  Cross 
checking  the  merge  data  is  performed  to 
identify  and  record  merge  recommendation 
conflicts. 

The  Operator  Query  mode  evaluates  the 
M-scores  for  a  given  CST  versus  its 
candidate  list  which  may  or  may  not  exclude 
tracks  derived  from  specified  sources.  The 
candidates  are  ranked  in  order  of  their 
merge  scores  even  if  the  merge  scores  do  not 
pass  the  merge  threshold.  The  maximum 
M-score  is  used  in  the  ranking.  For 
candidates  that  are  recommended  for  merge, 
the  merge  conflicts  are  identified  if 
existent . 

The  Delete  Tracks  mode  purges  the 
Candidate  Lists  and  Merge  Recommendation 
Data  of  tracks  that. have  been  deleted  from 
inventory  by  the  TDP.  The  Operator  Reject 
mode  updates  the  above  data  files  to  enforce 
merge  decisions  made  by  the  TDP  operator. 

DISCUSSION 

The  performance  capability  of  TAA  has 
been  tested  against  ground  truth  based  on 
simulated  track  data.  The  test  compared  the 
algorithm's  performance  to  that  of  several 
operators.  A  series  of  tests  applied  to 
archived  operational  data  is  presently  under 
way. 

An  ensemble  of  31  tracks  was  installed  oh 
the  Shore  Targeting  Terminal  System,  a 
derivative  of  the  AN/USQ-81 (V) .  The 
smoothed  tracks  were  plotted  on  the  geo 
display,  and  a  series  of  view  foils  were 
made  of  the  smoothed  tracks  and  their 
covariance  ellipses  ("slinkies”) .  Figure  3 


illustrates  a  sequence  of  position  contacts 
and  the  contact  areas  of  uncertainty  (AOUs) . 
In  Figure  4,  the  contacts  of  Figure  3  have 
been  smoothed  by  the  MTST  filter.  All  the 
tracks  in  the  ensemble  were  in  the  same  time 
period  so  that  time  overlap  was  always 
present.  Attributes  were  excluded  in  the 
test.  These  data  and  the  contact  reports 
were  given  to  operators .  They  were  asked  to 
pick  out  tracks  that  represented  the  same 
ground  truth.  The  results  of  their 
performance  are  given  in  Table  2. 

The  same  ensemble  of  tracks  was 
evaluated  by  the  TAA  algorithm.  The 
separation  distance  criterium  d,  was  varied 
over  the  set  (5,10,15,20)  NM,  and  the  MERGE 
THRESHOLD,  Mo,  was  given  two  values,  viz., 

99  and  98  per  cent  corresponding  to  Type  II 
errors  of  1  and  2  per  cent,  respectively. 

The  performance  results  are  shown  in  Table  3. 

As  is  shown  in  Table  3,  an  increase  in 
the  distance  parameter,  d,  improves  the 
detection  of  track  pairs  that  should  be 
merged,  but  at  the  expense  of  increasing  the 
number  of  incorrect  merges.  The  latter 
represents  the  Type  II  error.  A 
conservative  approach  especially  applicable 
to  automatic  processing,  where  manual 
evaluation  is  to  be  maintained  at  a  minimum, 
mandates  that  the  rate  of  incorrect  merges 
be  small,  while  depending  upon  future 
evaluations  (as  the  tracks  are  updated)  to 
correct  the  missed  merges.  A  low  rate  of 
incorrect  rejections,  the  Type  I  error,  is 
required  so  that  the  missed  merge  tracks 
will  automatically  be  resubmitted  for 
evaluation  upon  their  updating.  For  the 
test  ensemble  above,  the  incorrect  rejection 
rate  was  nil. 


TABLE  2 

PERFORMANCE  OF  OPERATOR  IN  CORRELATING  31  TRACKS 


OPERATOR 

EXECUTION 

CORRECT 

MISSED 

INCORRECT 

NUMBER 

TIME  (HRS) 

MERGES 

MERGES 

MERGES 

1 

1/4 

7 

17 

6 

2 

2/3 

15 

9 

4 

3 

1-1/3 

4 

20 

17 

4 

4 

18 

6 

11 
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TABLE  3 

PERFORMANCE  OF  ALGORITHM  IN  CORRELATING  31  TRACKS 


SEPARATION  MF,RGE  CORRECT  MISSED  *  INCORRECT 

CRITERIA  THRESHOLD  MERGES  MERGES  MERGES 


5  NM 

99  % 

10 

14 

0 

10  NM 

99  % 

16 

8 

0 

15  NM 

99  % 

19 

5 

3 

20  NM 

99  % 

24 

0 

8 

5  NM 

98  % 

13 

11 

0 

10  NM 

98  % 

16 

8 

1 

15  NM 

98  % 

21 

3 

3 

20  NM 

98  % 

24 

0 

8 

*  HOLDS 


CONCLUSIONS 

The  need  for  an  automatic  track 
association  algorithm  in  current  TDPs  has 
been  demonstrated.  With  as  few  as  30  tracks 
in  a  data  base,  an  operator  will  be  faced 
with  over  400  possible  pairwise  comparisons. 
With  a  larger  data  base,  the  task  is 
overwhelming  for  manual  solution. 

The  algorithm  that  was  developed  is 
based  on  a  statistical  comparison  between 
two  tracks.  In  contrast  with  other 
approaches,  it  uses  multiple  contact  data  of 
a- track. 

The  results  concerning  the  performance 
of  the  algorithm  indicate  that  it  is 
effective  in  filtering  out  pairs  of  tracks 
that  come  from  different  platforms. 

However,  there  still  appears  to  be  a  need 
for  an  operator  to  make  the  final  decision 
whether  two  tracks  should  be  merged. 

The  TAA  approach  to  implementation  of 
track  association  can  be  seen  as  an 
alternative  to  the  Bayesian  multiple 
hypothesis  systems  that  have  been  proposed. 
The  TAA  approach  assumes  that  a  series  of 
correlator-trackers  have  operated  on  each  of 
several  input  data  streams  to  form  tracks. 
The  function  of  TAA  is  to  sort  through  the 
sets  of  tracks  and  identify  those  that  are 
merge  candidates.  The  advantage  of  the  TAA 
approach  includes  its  implementation  in  a 
modular  form  in  a  TDP,  exploiting  the 
correlation  algorithms  that  are  in  place. 
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A  USER-SYSTEM  INTERFACE  DESIGN  TOOL 


Stephen  L.  Morgan  and  Alexander  Nauda 


HRB-Singer,  Inc. 
Box  60,  Science  Park 
State  College,  PA  16804 
(814)  238-4311 


Described  is  a  sophisticated  system  design  tool  for  rapid  protoyping  and  development  of  man/machine  interfaces  tailored  to  the  needs  of 
C^I,  I&W,  situation  analysis,  and  other  data  fusion  systems.  The  User  System  Interface  Design  Tool  (USIDT)  is  a  versatile,  interactive  software 
system  which  resides  on  a  Silicon  Graphics  workstation  with  an  external  map  database  and  scenario  data  files.  With  this  advanced  prototyping 
capability,  designers  can  develop  and  demonstrate  detailed  interactive  displays,  menus,  maps,  and  processing  sequences  early  in  the  software 
design  phase.  USIDT  capabilities  include  a  robust  display/menu  construction  tool  set,  a  map  database  and  scenario  file  retrieval  system,  a 
friendly  interface  for  inexperienced  users,  and  a  strong  simulation  capability  to  allow  end  users  to  examine  and  critique  the  menu  interaction, 
interactive  map  and  data  overlays,  and  zooming  on  the  map  displays  (before  any  production  software  is  generated).  Further,  USIDT  provides  the 
capability  to  collect,  analyze,  and  report  human  factors  data  not  available  elsewhere  in  such  a  framework. 

Prototyping  the  user  system  interface  allows  government  reviewers  to  examine  proposed  system  designs,  to  suggest  changes  to  the  struc¬ 
ture  or  flow  of  the  interaction,  and  to  observe  the  results  of  the  modifications.  The  more  precisely  a  design  can  be  shown  to  reviewers,  the  more 
likely  that  problems  will  be  identified  and  corrected  early  in  the  software  development  life  cycle,  thereby  reducing  total  system  cost  and  produc¬ 
ing  a  usable  system,  adhering  to  military  standards.  The  use  of  this  design  tool  will  improve  the  user  system  interfaces  of  data  fusion  systems  and 
will  reduce  development  costs  by  minimizing  late  software  life  cycle  rework  as  a  result  of  the  improved  communication  between  system  design¬ 
ers  and  government  reviewers  early  in  the  design  phase,  when  modification  of  the  design  is  easier  to  accomplish  and  less  expensive. 

The  use  of  USIDT  is  demonstrated  through  sample  menus  and  graphic  output.  Simulation  and  human  factors  data  collection  capabilities 
are  illustrated  with  examples  of  selected  display  sequence  simulations. 
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State  College,  PA  16804 


IfflRQDUCTiON 

With  the  availability  of  high  speed  data  links,  tracking 
sensors,  airborne  reconnaissance  devices,  satellite  data,  and  recent 
advances  in  EW,  there  is  a  staggering  amount  of  information 
available  to  the  C^I,  I&W,  and  situation  analysis  community. 
Although  high  speed  computers  can  rapidly  process,  catalog,  file, 
and  format  this  data,  it  is  the  user  interface  that  is  responsible  for 
presenting  this  information  clearly  and  concisely  to  tactical  and 
strategic  decision  makers  (Ref.  1).  While  the  availability  of  color 
CRT  or  raster  scan  hardware  has  created  new  options  for  the 
display  of  complex  data,  the  flexibility  offered  by  software  systems 
makes  possible  the  optimization  of  these  displays  to  meet  user 
requirements.  However,  the  needs  of  the  user  frequently  have  been 
overlooked  until  the  overall  system  has  been  developed,  leading  to 
increased  costs  due  to  rework  late  in  the  software  life  cycle.  This, 
and  the  increased  size  and  complexity  of  data  fusion  systems,  has 
resulted  in  software  costs  exceeding  hardware  costs  by  an  increas¬ 
ing  ratio  (Ref.  2,3).  Industry  experience  indicates  that  the  use  of 
software  development  tools  can  improve  productivity  by  identify¬ 
ing  unsatisfactory  elements  (for  example,  in  the  man/machine 
interface  design)  early  in  the  design  process  before  any  production 
code  is  generated,  rather  than  later  in  the  process  when  rework  and 
reimplementation  forces  costs  to  be  higher  and  results  in  delayed 
development  schedules  (Ref.  4,5).  Without  thorough  consideration 
of  the  way  data  is  displayed  to  decision  makers,  a  data  fusion 
system  may  be  impotent.  Rapid  prototyping  and  simulation  of 
user-system  interfaces  makes  possible  the  evaluation  of  design 
trade-offs  and  engineering  of  human  factors  into  the  system  early 
in  the  design  phase  when  user-requested  modifications  of  the 
design  are  easier  to  accomplish  and  less  expensive. 

Until  recently,  there  has  been  a  lack  of  software  develop¬ 
ment  tools  to  expedite  the  development  of  realistic  user-system 
interfaces  that  can  be  critiqued  and  examined  during  system  defini¬ 
tion  and  preliminary  design.  System  design  tools  for  user-system 
interfaces  need  to  produce  design  expressions  or  documentation  of 
sufficient  detail  and  scope  to  permit  government  reviewers  to  fully 
understand,  critique,  and  see  the  operational  consquences  of  a  user- 
system  interface  design.  Useful  tools  should  have  the  capability  to 


analyze  or  track  the  problems  a  user  experiences  when  interacting 
with  a  prototype  system.  Map  data  retrieval  and  display  capabili¬ 
ties  are  necessary  to  prototype  C%  I&W,  or  situation  analysis 
systems.  In  addition,  the  hardware  and  software  architectures 
selected  by  the  system  designer  may  place  constraints  on  the 
command  structures  and  the  data  a  user  sees  and  manipulates,  the 
logical  relationships  between  commands,  the  resulting  displays, 
and  the  responsiveness  of  the  system  to  user  requests.  For  the 
system  to  perform  as  required,  the  system  design  must  result  in  an 
environment  which  is  easy  to  use.  Thus,  user- system  interface 
protot>^es  developed  without  adequate  tools  are  often  clumsy. 

Yet,  a  good  user-system  interface  is  critical  for  full  and  easy  use  of 
system  functions. 

Deficiencies  in  user-system  interface  design  tools  are  par¬ 
ticularly  marked  in  systems  employing  high-resolution  color 
graphics  displays  and  menus,  where  text  and  traditional  pen  and  ink 
expressions  of  the  design  cannot  communicate  the  flow  of  the 
interaction,  the  “feel”  of  the  interactive  devices,  nor  the  visual 
impact  of  the  use  of  color.  Also,  these  systems  do  not  produce  the 
data  required  to  analyze  the  user’s  problems  in  interacting  with  the 
system.  A  reviewer  trying  to  evaluate  whether  an  interface  repre¬ 
sents  a  good  design  must  read  a  technical  description  of  the  menu 
hierarchy  and  the  meaning  and  consequences  of  each  command  and 
data  entry  field.  Even  with  a  comprehensive  verbal  description  of 
the  interface,  the  flow  of  the  interaction  is  difficult  to  grasp  and  the 
ergonomic  issues  of  color  and  interactive  device  utilization  are 
difficult,  if  not  impossible,  to  communicate.  Furthermore,  a  techni¬ 
cal  description  does  not  allow  the  human  factors  analyst  to  monitor 
and  analyze  data  produced  by  the  user  interacting  with  the  system. 

The  commercial  marketplace  offers  some  display  construc¬ 
tion  tools,  ranging  in  diversity  from  “paint”  programs  for  micro¬ 
computers  to  very  complex  image  processing  capabilities  in  sup¬ 
port  of  the  video  production  industry.  Although  these  tools  address 
graphic  construction,  they  do  not  incorporate  all  the  capabilities 
needed  to  produce  user-system  interface  prototypes,  particularly  for 
CT,  I&W,  situation  analysis,  and  other  data  fusion  systems. 
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EXISTING 

MMIST 


EXECUTIVE 

COMPONENT 


•SIMULATION  INITIALIZATION 
AND  TERMINATION 

•LOG  ON,  SIMULATION  SETUP 
AND  MAIN  MENUS 


•  FRAME  NAMING,  STORAGE 
RETIRIEVAL,  TIME  DELAY 
AND  EDIT  COMMANDS 

•  GRAPHICS  ELEMENT  CREATION, 
PLACEMENT,  AND  MANIPULATION 
COMMANDS 

•  TEXT  INPUT,  PLACEMENT 
AND  CHARACTERISTICS 
COMMANDS 

•  SYMBOL  CREATION  AND 
PLACEMENT  COMMANDS 


SYMBOL 

FRAME 


•  ESTABLISH 
CONNECTIVITY  NETWORK 
BETWEEN  FRAMES 

•  THREE  TRANSITION  ACTIONS: 
CURSOR;  KEYBOARD;  AND 
WAIT 

•  TWO  TRANSITION  MODES; 
REPLACEMENT  OR  OVERLAY 

•  DEFINE  SIMULATION  ENTRY 
POINT  AND  EXIT  TRANSITION 


SIMULATION 

EXECUTION 


•  EXECUTION 


•  SELECTION  OF  SIMULATION 
OPTIONS:  USE  OF  THE  DELAYS; 
CHANGE  START  POINT;  SAVE 
SEQUENCE  FOR  PLAYBACK 
MODE 

•  TWO  MODES  OF  EXECUTION: 
MANUAL  OR  PLAYBACK 


FRAME 

PLAYBACK 

CONNECTION 

SEQUENCE 

FILE 

SAVE  FILES 

•  EXPAND  SYMBOL  CAPABILITY, 
AND  ASSOCIATION  OF  SYMBOLS 
TO  POSITIONAL  DATA  RECORDS 


•TWO  NEW  TRANSITIONS 
MAP  AND  POSITIONAL 

•  DEFINE  MAP  AND 
POSITIONAL 
DISPLAY  OPTIONS 


•"REALTIME"  FRAME 
DISPLAY 

•  OUTPUT  HUMAN  FACTORS 
DATA  AND  ANALYSIS 
RESULTS 


Figure  1  USIDT  Built  On  Existing  MMIST  Capability 


To  improve  the  development  productivity  on  data  fusion 
systems  that  include  interactive  graphics  displays  in  their  man/ 
machine  interfaces,  HRB-Singer  developed  the  Man/Machine 
Interface  Simulation  Tool  (MMIST)  that  provided  the  capability  to 
construct  prototypical  menu  and  display  frames,  to  connect  the 
developed  frames  together  with  user  input  actions  (frame  transition 
actions),  and  to  execute  the  simulation  of  the  user  interaction  as  a 
sequence  of  frame  displays  interlaced  with  user  actions  (Ref.  6). 
This  tool  has  been  used  by  designers  and  government  design 
reviewers  to  develop  and  evaluate  prototype  displays,  menus,  and 
processing  scenarios  early  in  the  design  phase,  and  thus  ensure  the 
development  of  user-system  interface  software  packages  that  are 
usable  and  adhere  to  militaiy  standards.  A  significant  productivity 
gain  was  achieved  as  a  result  of  doing  a  better  job  of  detailing  and 
demonstrating  interactive  displays/menus  and  processing  se¬ 
quences  early  in  the  design  phase  (Ref.  6). 

This  paper  presents  HRB-Singer’s  newly  developed  User- 
System  Interface  Design  Tool  which  simultaneously  provides  the 
capabilities  to  build  prototypical  menu  and  display  frames,  define 
transitions  between  frames,  execute  a  simulation  (with  optional 
global  and  frame  specific  timing  delays),  retrieve  and  display  data 
from  external  map  and  scenario  data  files  as  part  of  the  simulation, 
and  monitor  user  interaction  with  the  simulation.  Built  upon  the 
existing  MMIST  software  package,  USIDT  adds  a  collection 
capability  for  user-system  interaction  data,  including  the  number, 


type  and  location  of  user  errors  and  analysis  of  user  command 
linkages,  frequencies,  and  trends  in  usage.  Further,  USIDT  adds 
the  capability  to  retrieve  and  display  map  and  scenario  data  from 
external  files  and  improves  the  flexibility  and  usability  of  the 
previous  MMIST  software. 

Use  of  USIDT  allows  early  insight  into  specification  of 
performance  requirements  and  identifies  performance  problems 
early  in  the  software  development  cycle.  To  the  DoD,  this  means 
that  USIDT  will  ultimately  reduce  costs,  reduce  system  mainte¬ 
nance,  increase  system  reliability,  and  improve  the  system’s 
usability. 

USIDT  OVERVIEW 

The  User-System  Interface  Design  Tool  (USIDT)  is  capable 
of  simulating  menu  interaction  and  interactive  map  and  data  over¬ 
lay,  including  simulation  execution  time  for  zooming  on  the  map 
displays.  Further,  USIDT  provides  the  capability  to  collect,  ana¬ 
lyze  and  report  human  factors  data  not  available  elsewhere  in  such 
a  framework.  The  use  of  this  design  tool  will  improve  the  user- 
system  interfaces  of  C^I,  I&W,  and  situation  analysis,  and  will 
reduce  development  costs  by  minimizing  late  software  life  cycle 
rework  as  result  of  the  improved  communication  between  system 
designers  and  government  reviewers  early  in  the  design  phase, 
when  modification  of  the  design  is  easier  to  accomplish  and  less 
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expensive.  Our  software  development  experience  has  shown  that  a 
software  function  modified  at  the  I&T,  or  delivery  phase  of  the 
software  life  cycle,  costs  about  70%  more  to  correct  than  the  same 
modification  made  during  the  design  phase.  Thus,  the  more  pre¬ 
cisely  a  design  can  be  shown  to  reviewers,  such  as  with  the  use  of 
the  USIDT  system,  the  more  likely  that  problems  with  the  user- 
system  interface  design  will  be  identified  and  corrected  early  in  the 
software  lifecycle,  thereby  reducing  total  system  development  cost 
and  producing  the  most  usable  system  possible. 

Architecture  Overview 

The  Man/Machine  Interface  Simulation  Tool  (MMIST) 
provided  the  capability  to  construct  prototypical  menu  and  display 
frames,  to  connect  the  developed  frames  together  with  user  input 
actions  (frame  transition  actions),  and  to  execute  the  simulation  of 
the  user  interaction  as  a  sequence  of  frame  displays  interlaced  with 
user  actions.  USIDT  was  built  upon  the  MMIST  capability  and  its 
software  architecture  continues  to  reflect  MMIST*s  structure. 
Figure  1  presents  the  four  major  components  of  the  MMIST  archi¬ 
tecture  and  the  additional  capabilities  incorporated  to  produce 
USIDT. 

The  Executive  Component  performs  simulation  initializa¬ 
tion  and  termination  tasks  in  addition  to  Logon,  Simulation  Setup, 
and  Main  Menu  interaction. 

Frame  Construction  provides  the  capabilities  with  which 
the  simulation  developer  builds  individual  simulation  frames 
including: 

-  naming,  storing,  and  retrieving  for  editing, 
deleting,  and  clearing  of  frames 

-  graphics  commands  for  placing  lines,  circles, 
rectangles,  ellipses,  polygons,  and  symbols  in 
frames 

-  symbol  construction  commands 

-  text  size,  font,  orientation,  input,  and  placement 
commands 

-  color  assignment  commands  for  text,  lines, 
closed  figure  interiors,  and  backgrounds 

-  utility  commands  to  remove  pictorial  elements 
from  frames,  copy  and/or  move  elements, 
reassign  colors,  fill  closed  figure  interiors, 
display  reference  frames,  display  user-specified 
reference  grid  (and  constrain  graphic  elements  to  the 
reference  grid),  and  to  redraw  the  frame  at  user  request 

-  set  frame-specified  time  delays 

Simulation  Construction  provides  the  capabilities  used  by 
the  simulation  developer  to  build  a  database  of  user  actions  (transi¬ 
tion  actions)  which  causes  the  simulation  to  advance  from  frame  to 
frame.  These  capabilities  include  commands  for: 

-  making,  editing,  or  breaking  linkages  between  simulation 
frames 

-  specifying  the  type  (cursor  placement,  keyboard  entry  or 
wait  time)  and  mode  (replacement  of  first  frame  by 
second  or  overlay  of  second  on  first)  of  user  action 

-  defining  the  starting  frame  of  the  simulation  and  for  defin 
ing  transition  actions  which  cause  the  simulation  to 
terminate 

-  displaying  and  outputing  the  characteristics  of  links 
between  frames 


Finally,  Simulation  Execution  provides  the  capabilities  to 
run  the  developed  simulation  and  to  select  various  execution 
options,  including: 

-  use  of  global  time  delay  values  and/or  the  frame 
specific  time  delays  assigned  to  each  frame  in  Frame 
Construction 

-  selection  of  a  different  starting  frame  than  the  one 
specified  in  Simulation  Construction 

-  requests  to  save  the  interactive  session  to  be  run  for 
playback  at  a  later  time 

-  requests  to  play  back  a  previously  saved  interactive 
session  and  the  selection  of  the  playback  execution  mode 

Added  Capabilities  of  USIDT 

To  develop  USIDT,  we  added  capabilities  to  our  MMIST 
tool  in  the  following  three  major  areas:  Human  Factors,  Map  and 
Scenario  Data  Display,  and  USIDT  User-System  Interface  En¬ 
hancements. 

o  Human  Factors 

-  Added  the  capability  to  record,  list,  and 
correlate  human  factors  data  (user  errors  and 
command  utilization  and  linkages) 

-  Upgraded  the  Simulation  Execution  capability 
to  allow  the  selection  of  human  factors  data 
collection  and  reporting 

o  Map  and  Scenario  Data  Display 

-  Added  the  capability  to  retrieve  and  display 
data  from  external  map  and  scenario  data  files 

-  Added  the  capaility  to  connect  or  track  multiple 
scenario  file  data  records  as  occurrences  of  the  same 
unit  over  time  during  simulation  execution 

-  Upgraded  the  Simulation  Setup  capability  to  allow 
users  to: 

— specify  the  location  and  extent  of  the  map  and  see 
nario  data  overlay  window  on  the  screen 
— specify  which  map  features  are  to  be  displayed  and 
the  color  assigned  to  the  displayed  features 
— select  the  type  of  symbology  to  bedisplayed  with 
scenario  data  records  (point,  ellipse,  track,  and/or 
symbol) 

-  Expanded  the  Symbol  Construction  functionality  by 
adding  a  capability  to  associate  symbols  with  records 
in  the  external  scenario  file 

-  Upgraded  the  Simulation  Construction  capability  to 
provide  for  new  user  transition  actions  allowing  users 
to  make  transitions  to  map  and  scenario  data  displays 

o  User  System  Interface  Enhancements 

-  Rehosted  the  VAX  1 1/780  FORTRAN  source 
code  to  the  Silicon  Graphics  3120  workstation 

-  Upgraded  the  Simulation  Construction  capability  to 
provide  for  new  user  transition  actions  allowing  users 
to  select  on  screen  objects  as  a  basis  for  transition  in 
addition  to  keyboard  input,  cursor  selection  of  screen 
rectangular  areas,  and  time  delays 

-  Enhanced  the  Symbol  Constmetion  capability 

by  increasing  the  number  of  symbols  simultaneously 
available,  and  improving  the  symbol  storage  strategy 
making  it  possible  to  use  symlx)Is  developed  for  one 
simulation  in  subsquent  simulations 
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Human  Factors  Report  for:  DIM 


Sun  Auk  30  16:15:13  1987 

Origin  Destination  Time  Of  Waiting  Transition  Transition  Transition 
Frame  Frame  Response  Time  Number  Type  Description 


Start  1 

1  Error 

1  2 

2  Error 

2  Abnorm  Term 


16:14:57  00:00:00 
16:15:02  00:00:04 
16:15:04  00:00:02 
16:15:06  00:00:02 
16:15:08  00:00:02 


Total  Transition  Request  Errors:  2 


Possible  Typographical  Errors 


Begin 

Simulation  Initiation 

Text 

GOTOO 

Text 

G0T02 

Region 

(  0.334,  0.342) 

Region 

<  0.750,  1.000) 

String  Possible 

Entered  Match 


GOTOO  G0T02 

End  Of  Human  Factors  Report  Generation. 

Tue  Sep  1  15:10:43  1987 


Origin  Destination  Time  Of  Waiting  Transition  Transition  Transition 
Frame  Frame  Response  Time  Number  Type  Description 


Start  1  15:09:17 

1  2  15:09:20 

2  3  15:09:21 

3  4  15:09:26 

4  1  15:09:32 

1  2  15:09:35 

2  3  15:09:36 

3  4  15:09:41 

4  1  15:09:47 

1  2  15:09:49 

2  3  15:09:51 

3  4  15:09:55 

4  1  15:10:01 

1  2  15:10:04 

2  3  15:10:06 

3  4  15:10:10 

4  1  15:10:14 

1  2  15:10:18 

2  3  15:10:20 

3  4  15:10:24 

4  1  15:10:28 

1  2  15:10:32 

2  3  15:10:34 

3  4  15:10:38 

4  Norm  Term  15:10:42 


00:00:00 

— 

Begin 

00:00:03 

2 

Text 

00:00:00 

3 

Region 

00:00:00 

4 

Time 

00:00:04 

5 

Text 

00:00:02 

2 

Text 

00:00:00 

3 

Region 

00:00:00 

4 

Time 

00:00:04 

5 

Text 

00:00:02 

2 

Text 

00:00:01 

3 

Region 

00:00:00 

4 

Time 

00:00:04 

5 

Text 

00:00:03 

2 

Text 

00:00:02 

3 

Region 

00:00:00 

4 

Time 

00:00:02 

5 

Text 

00:00:04 

2 

Text 

00:00:02 

3 

Region 

00:00:00 

4 

Time 

00:00:02 

5 

Text 

00:00:04 

2 

Text 

00:00:01 

3 

Region 

00:00:00 

4 

Time 

00:00:02 

6 

Text 

Simulation  Initiation 
G0T02 

(  0.000,  0.651)  (  0.488,  0.000 

Time  Delay  Transition 

G0TO3 

G0T02 

(  0.000,  0.651)  (  0.488,  0.000 

Time  Delay  Transition 

G0T03 

G0T02 

(  0.000,  0.651)  (  0.488,  0.000 
Time  Delay  Transition 
G0T03 
G0T02 

(  0.000,  0.651)  (  0.488,  0.000 

Time  Delay  Transition 

G0T03 

G0T02 

(  0.000,  0.651)  (  0.488,  0.000 

Time  Delay  Transition 

G0T03 

G0T02 

(  0.000,  0.651)  (  0.488,  0.000 

Time  Delay  Transition 

STOP 


Transition  Number  Sequence  Occurrence  Rates 


Occurrences  Sequence 


3  4  5 

2 

3 

4 
6 


Total  Transition  Request  Errors;  0 


End  Of  Human  Factors  Report  Generation. 


Figure  2  Sample  Human  Factors  Report 
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>  upgraded  the  Simulation  Setup  capability  to 
allow  users  to  interactively  modify  the  color  palette 

Reporting  Human  Factors  Data 

The  Simulation  Execution  component  now  records  and 
reports  human  factors  data.  This  allows  a  human  factors  analyst  to 
analyze  a  proposed  design  and  suggest  changes  based  on  data 
collected  during  system  prototyping.  In  MMIST,  the  transition 
between  any  pair  of  simulation  frames  is  controlled  by  the  user 
performing  a  transition  action,  such  as  typing  at  a  keyboard  or 
positioning  a  cursor  on  screen  and  pressing  a  button.  The  simula¬ 
tion  previously  had  a  database  of  all  permissible  transitions  be¬ 
tween  frames.  The  existence  of  this  data  made  it  possible  for 
USIDT  to  record  the  number  of  times  a  user  tries  to  perform  an 
undefined  transition  (e.g.,  enters  an  unknown  character  string  at  the 
keyboard  or  fails  to  place  the  on-screen  cursor  over  the  defined 
rectangular  areas  or  over  objects  for  which  transitions  exist).  The 
record  of  unsuccessful  transition  attempts  is  summarized  and 
characterized  to  identify  the  fi^uency  of  typographical  errors,  and 
attempts  to  perform  undefined  transitions.  This  record  may  then  be 
used  to  identify  those  transitions  which  perhaps  should  be  included 
and  that  the  prototype  user  expects  to  be  able  to  perform. 

In  addition  to  the  simulation  log  generation  and  reporting, 
we  installed  new  software  to  analyze  the  transitions  log  to  identify 
command  linkages  and  the  frequency  of  occurrence  of  the  repeti¬ 
tion  of  set  of  transitions.  Analysis  of  the  frequency  of  occurrence 
observations  is  used  to  identify  command  stractures  which  may  be 
awkward  and  need  to  be  restructured.  This  analysis  can  reveal  that 
some  commands  are  being  repeated  many  times  and  need  to  be 
examined  for  possible  simplification,  streamlining,  or  restructuring. 

Figure  2  presents  two  samples  of  the  human  factors  report. 
The  first  section  of  the  repct  is  a  log  of  the  transitions  performed 
containing  data  on  the  frame  from  which  the  transition  occurred, 
the  frame  to  which  the  simulation  advanced,  the  time  of  day,  the 
time  required  for  the  user  to  perfonn  the  transition,  the  record 
number  of  the  transition  in  the  permissible  transitions  data  base,  the 
type  of  transition  attempted,  and  a  transition  action  description, 
either  in  the  form  of  a  text  string,  normalized  screen  coordinates, 
object  Pick  Id,  or  user-specified  time  delay.  Following  the  simula¬ 
tion  log,  the  report  displays  the  list  of  any  multiple  occurrences  of 
the  transition  series  sets.  The  total  number  of  transition  errors 
made  during  the  simulation  execution  follows  the  list  of  transition 
sequence  repetitions.  If  any  typographical  errors  have  been  made, 
then  the  report  contains  a  section  identifying  the  character  strings 
in  error  and  the  possible  matches  in  the  permissible  transitions 
database. 

Displaying  Map  and  Scenario  File  Data 

The  prototyping  of  systems  utilizing  interactive  map  dis¬ 
plays  (C^I,  I&W,  and  Situation  Analysis)  require  zooming  and 
panning  on  user-selected  map  centers  and  at  user-selected  map 
scales.  User-system  interface  simulation  systems  which  do  not 
provide  for  zooming  and  panning  of  map  displays  can  not 
adequately  simulate  this  functionality  in  other  ways.  Further,  since 
the  display  of  data  against  the  map  is  the  critical  element,  prototyp¬ 
ing  tools  for  these  types  of  simulations  need  a  mechanism  for 
incorporating  scenario  data.  To  address  these  deficiencies  in  our 
MMIST  tool  (and  in  the  products  available  in  the  commercial 


marketplace),  we  have  incorporated  map  and  scenario  data  base 
retrieval  and  display  capabilities.  USIDT  simulations  are  able  to 
zoom  and  pan  on  map  displays  and  scenario  data  overlays  during 
simulation  execution,  and  thereby  provide  the  most  realistic  proto¬ 
type  of  the  system  possible. 

To  provide  for  the  simulation  of  interactive  map  and  sce¬ 
nario  data  display,  we  installed  external  map  database  and  scenario 
file  data  retrieval  and  display  capabilities.  To  provide  for  the 
realistic  treatment  of  scenario  data  display,  we  installed  a  capability 
to  associate  user-developed  symbols  with  scenario  data  for  the 
purpose  of  displaying  symbols  during  simulation  execution.  In 
addition  to  symbols,  scenario  data  may  be  optionally  displayed  with 
ellipses  of  locational  uncertainty,  geolocation  indicators,  identifica¬ 
tion  code  character  strings,  and  track  or  patrol  paths  for  moving 
units.  Since  scenario  data  is  often  dynamic  with  respect  to  time, 
we  installed  the  capability  to  place/move/remove  scenario  data 
display  symbology  on/over/from  the  map  background  as  the  sce¬ 
nario  data  file  indicates  the  movement  through  time  of  mobile  units. 

The  simulation  developer  creates  the  external  scenario  data 
file  outside  the  USIDT  tool  using  the  host  computer’s  text  editor. 

A  scenario  file  data  record  includes: 

-  An  identification  code  for  the  unit  (up  to  5  characters) 

-  Unit  mobility  code  (fixed,  hostile  track,  friendly  patrol 
path) 

-  Time  of  first  and  last  occurrence  of  the  unit 

-  For  mobile  units,  the  location,  time,  and  locational 
uncertainity  ellipse  parameters  for  each  point  in  the  track 
or  patrol  path 

Having  the  scenario  data  in  an  external  file  allows  the 
USIDT  user  to  easily  modify  simulation  scenario  data.  Thus,  new 
scenarios  can  be  developed,  experimented  with,  and  demonstrated 
without  modifying  the  simulation’s  internal  structure  in  any  way. 

The  external  map  database  incorporated  into  USIDT 
contains  1.5  million  points  in  six  categories: 

-  Shoreline 

-  Topographic  relief  contours  (8  levels) 

-  Bathymetric  depth  contours  (8  levels) 

-  Rivers,  lakes,  canals  and  seasonal  water  bodies 

-  Political  boundaries 

-  Cities  (names,  country  capitals,  and  four  levels  of 
population  size) 

This  map  database  was  constructed  by  direct  digitization 
from  the  Defense  Mapping  Agency  (DMA)  map  series  1150,  ‘The 
World.”  The  database  exists  in  four  levels  of  resolution  to  simulta¬ 
neously  support  map  detail  and  display  throughput  requirements. 
Series  1 150  maps  are  in  Mercator  projection,  and  cover  the  globe 
only  from  84‘’N  to  70“S.  The  polar  regions  of  the  world  were  filled 
in  from  equivalent  scale  maps  from  other  sources,  most  notably  the 
DMA  Polar  Regions  Atlas  and  National  Geographic  Maps  of  the 
Arctic  and  Antarctic.  Figure  3  presents  a  sample  of  the  map  data¬ 
base  incorporated  into  USIDT.  The  USIDT  user  has  control  over 
which  of  the  six  classes  of  map  features  to  be  used  in  simulation  of 
a  user-system  interface,  as  well  as  control  over  the  color  assigned 
to  each  feature  and  the  threshold  value  of  the  scale  at  which  the 
feature  first  appears. 
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Figure  3  Sample  Map  Data  Base-Great  Britian  and  Ireland 


Figured  Prototype 


C^I,  I&W  and  situation  analysis  systems  utilize  map  dis¬ 
plays  in  a  variety  of  ways  and  display  maps  in  a  variety  of  sizes 
and  locations  on  CRT  screens.  USIDT  provides  thirteen  combina¬ 
tions  of  map  window  sizes  and  locations.  These  combinations 
include: 

-  Full  screen  map  window 

-  Four  3/4  screen  (in  both  height  and  width)  map  windows 

-  Four  1/2  screen  (in  both  height  and  wiath)  map  windows 

-  Four  1/2  by  full  screen  (or  full  by  1/2)  map  windows 

Further,  USIDT  provides  the  simulation  developer  with 
capabilities  to  select  map  features,  colors,  fill  options  for  polygon- 
based  features,  feature  selection  based  on  scale  of  displayed  maps, 
and  the  type  of  symbology  to  be  displayed  for  each  scenario  data 
item.  Figure  4  presents  a  sample  of  a  USIDT  map  and  scenario 
data  display  created  by  the  user  selecting  the  following  options: 

-  Upper  left  comer  3/4  screen  map  window 

-  Fill  of  polygon  features 

-  All  map  features  except  bathymetric  depth 

-  Scenario  data  displayed  with  symbols  and  tracks 
(moving  units  only) 

-  User  specified  map  colors 

The  Simulation  Construction  component  of  USIDT  now 
permits  the  simulation  developer  to  connect  the  simulation  frames 
to  the  map  and  scenario  data  displays,  thereby  permitting  the  user 
to  develop  part  of  the  display  screen  as  table  displays  and  interac¬ 
tive  menus  and  part  of  the  display  screen  for  the  real-time  constmc- 
tion  of  map  and  scenario  data  displays. 

The  USIDT  User-Svstcm  Interface 

In  response  to  requests  from  users,  USIDT  has  been  en¬ 
hanced  in  a  number  of  other  ways  in  order  to  increase  speed  in 
retrieving  complex  frames,  interactively  define  color  palettes, 
increase  symbols,  and  increase  user  transition  selection.  This 
allows  the  building  of  more  robust  simulations  and  results  in  not 
only  more  timely  prototypes,  but  also  more  effective  evaluation  of 
proposed  designs  by  DoD  reviewers  at  less  cost.  Perhaps  the  most 
important  of  these  additional  enhancements  was  the  porting  of  the 
software  from  the  Ramtek  9465/VAX  1 1/780  configuration  to  a 
Silicon  Graphics  workstation.  The  former  configuration  suffered 
from  an  inadequate  graphics  engine  coupled  with  an  overburdened 
minicomputer  with  the  result  that  MMIST  was  painfully  slow  at 
retrieving  and  displaying  complex  simulation  frames  and  was  slow 
to  react  to  user  commands.  The  graphics  engine  of  the  Silicon 
Graphics  3120  is  at  least  ten  times  faster  at  performing  graphics 
instructions  than  the  Ramtek.  The  Silicon  Graphics  has  approxi¬ 
mately  the  same  computation  power  as  the  VAX  11/780  and  is 
dedicated  to  the  USIDT  user  rather  than  shared  by  many  VAX 
users. 

Other  user-requested  enhancements  included  a  capability  to 
interactively  redefine  the  color  palette  available  to  the  simulation 
frame  developer,  and  to  provide  for  the  development  and  use  of 
more  symbols.  MMIST  allowed  the  user  only  14  simultaneous 
symbols.  These  symbols  were  stored  as  a  group  and  were  very 
difficult  to  reuse  in  subsequent  simulations.  USIDT  now  permits 
the  user  to  have  99  simultaneously  available  symbols.  Each  sym¬ 
bol  is  stored  as  a  discrete  unit  and  much  more  transportable. 

USIDT  provides  interactive  color  palette  modification  allowing  the 
simulation  developer  to  modify  any  or  all  of  the  colors  in  the  78- 
color  palette.  For  example,  if  a  simulation  under  development 


requires  10  grey  scales  (the  USIDT  default  color  palette  contains 
only  6  grey  scales),  the  simulation  developer  may  now  elect  to 
reassign  any  four  other  colors  to  the  required  grey  values. 

To  further  improve  the  flexibility  of  the  simulation  of  the 
user-system  interface,  an  additional  type  of  user  transition  action 
has  been  added  to  three  previously  existing  transition  formats 
( keyboard  input,  occurrence  of  a  cursor  button  push  in  rectangular 
subregions  of  the  screen,  and  wait  times).  This  new  user  transition 
action  is  cursor  selection  (by  Pick  Id)  of  objects  displayed  on  the 
screen.  Often  users  select  commands  via  on-screen  menus.  For 
this  type  of  user  action,  the  rectangular  area  selection  by  cursor  is 
appropriate.  Perhaps  nearly  as  often,  users  point  to  or  select  an 
object  for  an  additional  data  display.  For  this  type  of  action,  an 
object-oriented  transition  is  more  appropriate  than  the  rectangular 
area  transition. 

SUMMARY 

Rapid  prototyping  of  user-system  interfaces  helps  solidify 
system  requirements  and  identify  many  of  the  defective  or 
operationally  awkward  portions  of  a  design  which  would  otherwise 
go  undected  until  installation  of  the  completed  system.  Prototyping 
the  user-system  interface  allows  government  reviewers  to  examine 
the  proposed  system  design,  to  suggest  changes  to  the  structure  or 
flow  of  the  interaction,  and  to  observe  the  results  of  the  modifica¬ 
tions.  The  more  precisely  a  design  can  be  shown  to  reviewers, 
such  as  with  the  use  of  our  USIDT  system,  the  more  likely  that 
problems  with  the  user-system  interface  design  will  be  identified 
and  corrected  early  in  the  software  life  cycle,  thereby  reducing  total 
system  development  cost  and  producing  the  most  usable  system 
possible.  Further,  prototyping  makes  it  possible  for  system  design¬ 
ers  to  collect  and  analyze  data  on  the  user’s  interaction  with  the 
prototype  expression  of  the  design.  The  prototyping  of  the  C3I, 
I&W,  and  situation  analysis  systems  utilizing  interactive  menu, 
map,  and  scenario  data  overlays,  need  to  be  able  to  pan  and  zoom 
on  map  and  data  displays,  and  this  is  only  possible  when  map  and 
data  files  are  accessible  to  the  simulation.  HRB-Singer’s  USIDT  is 
the  most  complete  user-system  interface  prototyping  tool  available 
today. 
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TRACK  FILE  FUSION 
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ABSTRACT 

In  general,  the  most  accurate  method  of  data  fusion  (in  the  Minimum  Mean-Squared  Error  (MMSE)  sense)  is  to  receive  uncorrelated,  linear  data  and  fuse 
it  with  a  Minimum  Mean-Squared  Error  filter.  In  practice,  however,  the  data  rate  and  the  volume  of  measurement  data  available  in  a  multispectral  sensor 
system  are  so  high  that  operating  a  MMSE  filter  on  every  available  data  sample  is  an  unreasonable  solution  to  the  data  fusion  problem.  Instead,  it  may  be  more 
feasible  for  a  fusion  processor  to  accept  filtered  data  and  to  generate  a  suboptimal  estimate.  In  this  paper,  we  develop  and  analyze  two  methods  to  fuse  filtered 
data.  Each  method  operates  on  the  track  files  generated  by  individual  sensor  subsystems.  The  first  method  accepts  track  files  and  their  error  covariances;  then 
assuming  that  the  track  files  from  different  sensors  are  uncorrelated  and  that  every  sensor  supplies  a  track  file  at  every  update  interval,  the  algorithm  calculates 
the  MMSE  fused  estimate  for  each  target.  The  second  method  is  an  extension  of  the  first;  it  accounts  for  the  cross-correlation  between  different  sensor 
estimates. 

The  two  track  file  fusion  methods  developed  in  this  paper  have  been  published  previously  by  other  authors.  We  have  derived  them  again  for  completeness 
and  run  simulations  to  analyze  their  performance.  In  this  paper,  we  present  comparisons  of  the  performance  of  the  track  file  techniques.  We  developed  an 
ideal  simulation  that  generates  individual  position-velocity  track  files  from  independent  Cartesian  position  measurements.  The  simulation  fuses  the  track  files 
with  each  of  the  techniques  discussed  above,  and  we  present  the  final  tracking  errors  on  a  single  plot  for  comparison. 
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Aerospace  Electronic  Systems 
Utica,  New  York  13503 


1.0  INTRODUCTION 

The  following  paper  originated  from  a  very  specific  question  about  a 
very  specific  problem.  While  attempting  to  fuse  filtered  estimates  from 
two  different  sources,  we  questioned  the  significance  of  the  cross-correla¬ 
tion  term  in  the  computation  of  the  fusion  gain.  In  the  process  of  deter¬ 
mining  the  cross-correlation  term’s  significance,  we  raised  a  number  of 
other  questions  about  the  fusion  of  two  track  estimates  and  about  hision 
in  general.  As  a  result,  we  wrote  the  following  paper  for  our  own  refer¬ 
ence.  It  has  become  a  tutorial  that  discusses  a  few  fusion  techniques  for 
kinematic  parameters.  Specifically,  we  discuss  the  significance  of  the 
cross-correlation  terms  in  the  gain  calculation.  This  paper  is  noi  a  discus¬ 
sion  of  the  accuracy  improvement  provided  by  data  ^sion.  In  our  experi¬ 
ence,  when  two  sensing  systems  measure  different  target  parameters 
(e.g.,  a  radar  system’s  range/range  rate  data  combined  with  an  infrared 
system’s  azimuth/elevation  data),  data  fusion  offers  unquestionable  im¬ 
provement  in  track  accuracy.  For  a  more  detailed  discussion,  we  refer¬ 
ence  the  reader  to  R.  Yannone’s  paper  on  sensor  fusion  [Ref.  4]. 

2.0  FUSION 

Fusion  is  such  a  general  topic  that  everyone’s  definition  and  applica¬ 
tion  is  a  little  different.  Before  we  begin  any  discussion  of  fusion,  it  is  best 
to  give  our  definition  of  the  term.  GE  Aerospace  Electronic  Systems 
(AES),  located  in  Utica,  New  York,  designs  airborne  sensors  (Infrared 
Search  and  Track  Systems  (IRSTS),  fighter  and  surveillance  radar,  elec¬ 
tronic  countermeasures  (ECM) ,  and  electronic  counter-countermeasures 
(ECCM)  receivers);  therefore,  all  of  our  fusion  applications  apply  to  air¬ 
borne  detection,  tracking,  and  identification  problems.  Inputs  to  the  fu¬ 
sion  process  could  be  raw  kinematic  measurements  consisting  of  range, 


range  rate,  azimuth,  and  elevation  measurements;  or  attribute  measure¬ 
ments  like  JEM-line  discriminations,  emitter  frequencies,  infrared  signa¬ 
tures,  etc.;  or  the  data  could  be  filtered  data  from  both  on-board  and 
off-board  assets.  Figure  1  illustrates  the  origins  of  the  airborne  data. 

We  define  the  fusion  processor  as  the  function  that  accepts  data  from  a 
variety  of  sources  and  combines  them  to  produce  one  composite  track 
file  on  each  target.  The  composite  track  file  has  better  track  accuracy 
and  higher  confidence  identification  on  each  target  than  any  one  sensor 
system.  In  most  systems,  the  composite  track  file  is  passed  on  to  an  infor¬ 
mation  processor  that  assesses  the  overall  battle  situation  based  on  the 
composite  track  files  and  a  priori  mission  database,  and  from  this  assess¬ 
ment,  recommends  a  response  or  allocates  the  system  resources  in  an 
optimal  manner.  The  following  discussion  applies  to  one  small  part  of  the 
fusion  processor.  It  does  not  attempt  to  discuss  possible  situation  assess¬ 
ment  or  response  techniques. 

We  break  the  fusion  processor  itself  into  three  subfunctions: 

•  Association/Correlation. 

•  Kinematic  fusion. 

•  Attribute  fusion. 

Figure  2  illustrates  these  subfunctions. 

The  Association/Correlation  subfunction  accepts  the  incoming  mea¬ 
surement  or  filtered  estimate  and  tags  it  to  the  appropriate  composite 
track  file  or  files.  (If  there  is  a  large  uncertainty  as  to  which  target  gener¬ 
ated  the  measurement,  then  a  probabilistic  association  of  the  measure¬ 
ment  to  multiple  track  files  is  probably  the  best  association  technique.) 
Once  we  have  correlated  the  measurement  to  a  composite  track  file,  the 
Kinematic  Fusion  subfunction  updates  the  track  file  kinematics  with  the 


Figure  1 .  Airborne  Sensor  Fusion  Problem 
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ON-BOARD 
SENSOR  SUITE 


Figure  2.  Top-Level  Fusion  Definition 


kinematic  measurement  parameters  via  a  Kalman  filter,  and  the  Attribute 
Fusion  subfunction  employs  a  probabilistic  or  expert  system  technique  to 
update  the  track  file’s  attribute  vector.  The  lower  portion  of  Figure  2 
illustrates  how  each  of  these  subfunctions  affects  the  composite  track  file 
assuming  the  measurement  originates  from  an  IR  sensor.  The  Associa¬ 
tion/Correlation  subfunction  appends  the  measurement  data  to  the  com¬ 
posite  track  file.  The  Kinem^c  Fusion  function  updates  the  file’s  kine¬ 
matic  state  vector  estimate  (X)  and  associated  error  covariance  matrix 
(P)  with  the  azimuth  (tt)  and  elevation  (e)  measurement.  The  Attribute 
Fusion  subfunction  updates  the  track  file’s  Friend/Foe/Neutral  (F/F/N), 
class  matrix  (CLASS)  and  aircraft  platform  identification  (A/C)  with  the 
new  IR  signature  information.  In  the  following  discussion,  we  will  concen¬ 
trate  on  the  Kinematic  Fusion  subfunction.  We  will  assume  that  the  Asso¬ 
ciation/Correlation  subfunction  performed  perfectly  and  tagged  the  mea¬ 
surement  to  the  correct  composite  track  file. 

There  are  really  two  types  of  kinematic  fusion:  measurement  fusion 
and  track  file  fusion.  The  difference  is  in  the  type  of  input.  If  the  incom¬ 
ing  data  is  uncorrelated  in  time  and  from  sensor  to  sensor,  then  we  em¬ 
ploy  the  measurement  fusion  technique.  If,  however,  the  data  is  filtered 
by  the  sensor  and  therefore  is  definitely  time-correlated  and  perhaps 
even  sensor-to-sensor-correlated,  then  we  use  the  track  file  fusion  tech¬ 
nique.  The  major  points  that  we  discuss  in  this  paper  deal  with  track  file 
fusion,  but  because  of  its  similarity  to  measurement  fusion,  we  discuss 
measurement  fusion  as  an  introduction  to  track  file  fusion. 


2.1  Measurement  Fusion 

The  measurement  fusion  algorithm  accepts  raw  uncorrelated  measure¬ 
ments  from  the  sensor  suite  and  uses  them  to  update  the  composite  track 
file  kinematics.  Assuming  that  we  require  a  linear  update,  we  can  express 
the  updated  estimate  as  a  linear  combination  of  the  previous  estimate  and 
the  incoming  measurement  vector, 

/V  A  ^ 

^(k/k)  =  Ki  •'X(k/k-l)  +  K2  •^i(k),  (1) 

where  X(k/k-l)  is  the  nxl  kinematic  state  vector  estimate  valid  at  t^ 
given  data  up  to  time  tk-i,  Zj  (k)  is  the  mjxl  measurement  vector 
generated  by  the  ith  sensor,  and  K  i  and  K  2  are  the  nxn  and  nxm  j 

linear  gains.  Our  objective  is  to  determine  K  and  K2  so  that  X(k/k)  is 
an  unbiased,  minimum  mean-square  error  estimate  of  the  true  kinematic 
state  vector,  X(k). 

We  begin  by  looking  at  the  error  terms  associated  with  equation  1. 
Assume  that  the  measurement  contains  additive,  white  noise,  then 

■^(k)  =  Ci  •Xik)  (k),  (2) 

where  C j  is  a  mj  xn,  linear  transformation  from  state-space  to  measure¬ 
ment-space,  and  iTi  (k)  is  the  additive,  zero-mean,  white  noise  vector. 


The  two  estimates  in  equation  1  may  be  expressed  in  terms  of  their  corre¬ 
sponding  errors, 

■X(k/k-l)  =  X‘(k)  -h"e(k/k-l),  (3) 

and 


A  ^ 

^(k/k)  =  X(k)  +t(k/k).  (4) 

Substituting  these  expressions  for  the  estimates  and  the  expression  for  the 
measurement  vector  into  equation  1,  we  get 

■X(k)  +t(k/k)  =  K  jiX(k)  +T(k/k-l)]  + 

K  2  •  [Ci  *X(k)  +iri(k)]. 

or  the  error  at  t  ^  gjven  the  data  at  t  k  is 

T(k/k)  =  (Ki  +  K  2  Ci  -  I)'X(k)  +  K  i"e(k/k-l)  -H  K  2  (k). 

(6) 

If  we  require  an  unbiased  estimate,  and  we  do,  then 

Ki  +  K2  •  Ci-  I  =  10],  (7) 


or 

Ki  =  I-K2*Ci.  (8) 

For  convenience  let 

K2  =  K,  (9) 

then 

Ki=:I-K*Ci  (10) 

Now  substitute  these  expressions  for  K  1  and  K  2  into  equation  6  and  find 
that  the  unbiased  error  is 

t(k/k)  =  (I  -  K  •  Ci)  •t(k/k-l)  +  K  .t* i  (k)  (11) 


We  have  to  solve  for  the  gain,  K,  that  minimizes  the  error  vector  e(k/k) 
at  every  time  t  k- 

We  minimize  the  total  length  of  the  error  vector, T(k/k) ,  by  minimizing 
the  sum  of  the  squares  of  its  individual  components.  Because  "t^k/k)  is 
stochastic,  we  choose  to  minimize  the  expected  value  of  the  squared  er¬ 
rors.  Thus,  set  the  gradient  of  E[^  (k/k)  ttk/k) ]  with  respect  to  K  equal 
to  zero,  then  solve  for  K. 


Vj,  {Eirrt]}  =  [0],  (12) 

where  we  let'^=*e(;k/k)  for  convenience,  and  Vr  is  the  matrix  gradient 
operator. 
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with  properties 

(vTrW)  =  v'wT  (14) 

and 


E  lu,(k)  .TT (k/k-1)]  =  [0],  (24) 

and  the  equation  we  must  solve  for  K  becomes 

K  {  C.  E  (€?k/k-l)  TT(k/k-l)]  C  .T+  E  l"?(k)  ujT  (k)]} 

=  E  (’e^k/k-l)"?!'  (k/k-1)]  cT.  (25) 

Let  the  error  covariance  matrbc  be 

P(k/k-l)  =  E  (■t(:k/k-l)'tT (k/k-1)]  (26) 

and  the  measurement  noise  covariance  matrix  be 

Rj(k)  =E  (Tj:(k)-^T(k)j,  (27) 

then  equation  25  becomes 


(VT  kT  K  W)  =  K  (W  WT)  (1 

From  the  previous  expression  for  €(k/k),  equation  11,  we  can  write 
T  =“e(k/k-l)  +  K  •  [v  j  (k)  -  C  .  •  'e(k/k-l)J  (L 

thus,  the  equation  we  must  solve  for  K  is 

\  (k/k-1)  +  {Vj'T  (k)  -‘TT(j^/k-l)  C  jTjkT} 

^  ^  ^  \1 
•  {€(k/k-l)  +  K  [b  j  (k)  -  Cj  €(k/k-l)]})  r:  [0] 


|E{'^^(^^*^-l)"€(k/k-l)  +*1^ (k/k-1)  Kiv‘.(k)  -  C.  ~€(k/k-l)] 
+  [^r.^  (k)  (k/k-1)  C.T]  K^T (k/k-1) 

+  (k)  -“€^ (k/k-1)  C.'^]  k'^K  [■ir.(k) 

-  C,t(k/k-l)]  }j=  [0]  (18) 

Employing  the  properties  listed  in  equations  14  and  15,  and  noting  that 
\  (k/k-l)"!"  (k/k-1)]}  =  [0],  (19) 

allows  us  to  perform  the  gradient  operation  in  equation  18  and  get 
2E  {■€(k/k-l)  .  [^T(k)  -“eT (k/k-1)  C.T] 

+  K  •  [u(k)  -  Crtik/k-l)]  •  tv.T(k)  -“TT (k/k-1)  C.T ]}  =  [0] 


where  we  have  made  use  of  the  fact  that  the  products  in  equation  18  are 
scalars  so  that 


T’^(k/k-l)  K[U(k)  -  cre(k/k-l)] 

=  fu.T(k)  -"tr  (k/k-1)  C.T]  KTt(k/k-l) 


Multiplying  the  terms  out  in  equation  20,  we  get 


E  ft^k/k-l)  .  ^^(k)]  -  E  [■f(k/k-l)  .-fr(k/k-l)]  C.T 

+  K  EiiT/k)  "uT/k)]  -  K  E(lr.(k)"rr (k/k-1)]  C.T 
-  K  C,  Ell(k/k-l)  uTlk)]  +  K  C.  Ejt/k/k-l)  "IT (k/k-1)]  C.T 


Recall  the  assumption  that  the  measurement  noise  is  zero— mean  and 
white.  Because  of  this  assumption,  the  measurement  noise  at  (k), 

must  be  uncorrelated  with  the  estimate  error  based  on  measurements 
occurring  through  tk-i  ,‘€(k/k-l).  Therefore 

E  ir(k/k-i)  .■uT(k)]  =  (0], 


K  {  C.  •  P(k/k-l)  •  C.T  +  R.(k)}  =  P(k/k-l)  •  C.T  (28) 
Solving  for  K,  we  find  our  optimal  measurement  fusion  gain  is 

(29) 

where  Ci  and  Ri(k)  depend  on  the  characterikics  of  the  ith  sensor. 

Now  that  we  know  what  K  is,  we  can  use  it  to  fuse  the  incoming  sensor 
measurement  Zi(k)  with  the  correlated  kinematic  state  vector  estimate 

X(k/k-l).  Applying  the  relationships  we  found  for  Kj  and  K2  to  the 
jSl 

expression  for  the  fused  track  file  X(k/k)  (equation  1),  we  have 

(30) 

This,  combined  with  equation  29,  makes  up  our  measurement  fusion  al* 

gorithm.  The  only  problem  is  that  we  have  the  unknowns  X(k/k-l)  and 
P(k/k-l). 

The  first  unknown  is  the  predicted  state  vector  estimate  given  data 
from  tk-i  .  If  the  true  state  vector  equation  is 

X(k)  =  +  B  •■tr(k-l)  +  G  •‘^(k-l)  (31) 

where  <I>  is  the  state  transition  matrix  from  tk-l  to  tk.  B  •‘u(k-l)  is  the 
deterministic  forcing  function,  and  G«‘^k-1)  is  zero-mean,  white  noise 
process  noise;  then  the  best  prediction  of  the  state  estimate  at  time 
tk-i  to  time  tk  is 

^k/k-1)  =  <l>>'^k-l/k-l)  +  B  •l5‘(k-l). 

The  error  associated  with  this  prediction  is 
t(k/k-l)  =  <[)  •“Ifk-l/k-l)  -  G  •'^(k-l), 
thus,  the  error  covariance  matrix  is 
P(k/k-l)  =  E  {[Ottk-l/k-1) 

-  G-^(k-l)]  [tV-l/k-D  ^  ]} 

or 

P(k/k-l)=^>*  Erltk-l/k-l)“e  ^(k-l/k-1)]  m 

-  G  •  E[^(k)  “eV-l/k“l)]  • 

-  O  •  E'i6(k-l/k-l)  ■w'r(k-l)]  •  G’T 
+  G  •  EKk-l)^(k-l)]  •  gT 

The  cross-terms  in  equation  35  equal  zero  because  the  error  contains  no 
data  from  time  tk»  and  data  from  time  tk-i  contains  ^(k-2),  not 
w(k-l).  Therefore,  “w(k-l)  and"l‘(k-l/k-l)  are  uncorrelated.  If  we  let 

p(k-i/k-i)  =  Ei€(k-i/k-i)'r^(k-i/k-i)]  (36) 

and  the  process  noise  covariance  matrix  be 


(32) 

(33) 

(34) 
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Q(k)  =  E[w(k-l)~w^(k-l)], 

then  the  predicted  state  estimate’s  error  covariance  matrix  becomes 


P(k/k-l)  =  0  •  P(k-l/k-l)  •  +  G  •  Q(k-l)  •  G 


(38) 


where  the  only  remaining  unknown  is  P(k-l/k-l)  or  the  error  covariance 
valid  at  the  same  time  as  the  data,  P(k/k).  We  find  P(k/k)  from 

P(k/k)  =  ET€(k/k]t^(k/k)]  (39) 

Substituting  the  equation  for  €(k/k)  (equation  11)  into  the  above  expres* 
sion  for  P(k/k),  we  find  that 

P(k/k)  =  E  {[(I-KCj)  i?k/k-l) 

+  K  U(k)l  [■tT(k/k-l)(I-K  C.  )T  +  ir.T(k)KT)} 

Recalling  that  the  cross-terms’  expected  values  equal  zero  (equations  23 
and  24),  we  have 


P(k/k)  =  (I-K  C.  )  •  E|€(k/k-l)"tT  (k/k-1)]  •  (I-KC.  )'^ 

+  K  .  Elir.(k)  ir.T  (k)l  •  kt 


(41) 


which  simplifies  when  we  recall  the  expressions  for  P(k/k-l)  and  Rj  (k) 
(equation  26  and  27). 


P(k/k)  =  (I  -  K  C.  )  .  P(k/k-l)  .  (I  -  KC,  K  R.(k) 


(42) 

After  substituting  for  K  in  this  equation  and  application  of  the  matrix 
inversion  lemma,  we  can  show  that 


P(k/k)  =  (I  -  KC.  )  •  P(k/k-l) 


(43) 


which  completes  our  derivation  of  the  measurement  fusion  filter. 

The  entire  recursive  algorithm  is  (from  equations  32,  38,  29,  43,  and 
30,  respectively) 


^(k/k-l) 

=  <[)*'^k-l/k-l)  -l-B 

•“u(k-l) 

P(k/k-l) 

=  0)  •  P(k/k-l)  •  ^  + 

G  •  Q(k-1)  •  G'^ 

K  =  P(k/k-l)cT  •  [C.P(k/k-l) 

C.T  +  R.(k)]"^ 

1  r 

P(k/k)  = 

(I-KCp  •  P(k/k-l) 

■^k/k)  = 

^k/k-1)  -h  K  •  IZ.(k) 

-C.  ."^k/k-l)] 

(44) 

(45) 

(46) 

(47) 

(48) 


Notice  that  this  is  the  exact  form  of  the  Kalman  filter  except  that  the 
state-space/measurement-space  transformation  Ci,  the  measurement 
covariance  Ri(k),  and  of  course  the  data  itself ^i(k)  aU  vary  as  a  func¬ 
tion  of  the,  sensor  that  generated  Z  j  (k) .  Figure  3  illustrates  the  algorithm 
in  block  diagram  form.  Next  we  show  how  this  form  differs  from  the 
track  file  fusion  case. 

2.2  Track  File  Fusion 

Processing  every  measurement  through  the  measurement  fusion  algo¬ 
rithm  discussed  above,  many  times  due  to  unavailability  of  data  or  pro¬ 
cessing  restrictions,  is  infeasible  for  each  measurement  on  every  target.  In 
these  cases  we  must  fuse  data  that  have  been  filtered  to  some  degree.  As 
a  first  step  in  developing  an  algorithm  to  perform  this  task,  let  us  examine 
the  two-sensor  problem  illustrated  in  Figure  4.  Both  sensors  generate 
their  own  estimates  of  the  target’s  kinematic  state  vector.  Assuming  that 
we  have  already  correlated  the  estimates,  what  is  the  linear,  unbiased, 
minimum  mean-squared  error  estimator  that  combines  the  filtered  data 
into  one  composite  track  estimate? 

The  problem  is  very  similar  to  the  measurement  fusion  problem.  We 
begin  by  requiring  a  linear  combination  of  the  two  sensor  estimates. 


1^k)  =  K  ^•^^(k/n)  +  K  2  •  X2(k/m), 


(49) 


where  X(k)  is  the  composite  estimate,  Xi(k/n)  is  the  first  sensor’s  esti¬ 
mate  of  X(k)  at  time  tj^  given  data  at  time  tn.  X2(k/m)  is  the  second 
sensor’s  estimate  of  X(k)  at  time  given  data  at  time  tmi  and  Ki  and 
K2  are  the  linear  gain  matrices.  As  in  the  measurement  fusion  case,  we 
find  an  expression  for  the  error  in  X(k).  If  we  express  the  errors  in  the 
estimates  as 


“X(k)  =’X(k)  +“e(k), 


^(k/n)  =^(k)  +'€i(k/n). 


and 


(50) 

(51) 

(52) 


;^(k/m)  =  CX(k)  e2(k/m), 

where  we  have  decided  to  estimate  the  composite  state,  f(k)  ,  in  the 

same  space  as  the  first  sensor’s  estimate,  ‘Xi(k/n),  then  following  the 
previous  measurement  fusion  derivation 

■X(k)  +-€(k)  =  Kj.  (X(k)  +-ej(k/n)l  +  (dc(k)  +"€2 

(53) 
or 

t(k)  =  (K  K  2  C  -  I)  -^(k)  +  K^«'€‘j(k/n)  +  K  ^  •^^(k/m). 

(54) - 
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Figure  3.  Measurement  Fusion  Solution 
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■  Objective;  Linearly  combine  and  X2  to  produce  a  MMSE,  unbiased  estimate  of  X 


Figure  4.  Track  File  Fusion  Problem 


where  we've  included  the  transtu.  nation  C  to  account  for  the  possibility 
that  the  first  and  second  sensors'  estimates  may  be  in  different  spaces. 
Assuming  the  individual  sensor  estimates  are  unbiased  (i.e.,  E[(ri(k/n)] 
=  E(€2(k/m)  -0),  we  require 


Ki  -f-  K2  C  =  I 

(55) 

for  an  unbiased  composite  estimate.  Let 

K2  =  K 

(56) 

then 

Ki  =  I  -  KC 

(57) 

and  our  composite  state  estimate  error  becomes 

T(k)  =  (I-KC)  •‘I'j  (k/n)  +K  (k/m) . 

(58) 

^  We  want  to  solve  for  the  matrix  K  that  minimizes  the  mean  length  of 
€(k)  at  every  time  instant.  Once  again,  we  want 


Vj^  E  [€T(k)-?(k)]  =  (0).  (59) 

Writing  the  error  as 

“e(k)  =Tj (k/n)  +  K[T2(k/m)  -  cT^ (k/n)]  (60) 

and  evaluating  equation  59,  we  have 

\  [E((^7(k/n)  +  (^^(k/ni)  -fTck/n)  cTj  KT){t;(k/n) 

1  M 

+  K  fe^Ck/m)  -  C  €j(k/n)|} )J  =  [0] 
or 

\  (e  {rj^(k/n)Tj(k/n)  +  Tj(k/n)  K  \t^(k/m)  -  cT^(k/n)] 

+  [Tj(k/m)  -Tj(k/n)  C^]  K^Tj(k/n) 

+  ^^(k/m)  -~t^(k/n)  K  [€  ^(k/m)  -  CTjCk/n)]}) 

=  [0]  (62) 

Evaluating  the  derivatives  first  gives  us  (see  equations  13-22) 

2  {e  I?j(k/n)  -  E  i?j(k/n)  T^V/n)]  c’^} 

+  2  K  {e  (fjCk/m)  T^(k/m)]  -  E  [t  j  (k/m)  Tj^Ck/n)]  C 
-  C  E  ^j(k/n)  +  C  E  [?j(k/n)  T jV/n))  } 

=  I0|-  (63) 

This  is  where  measurement  and  track  file  fusion  differ.  In  the  measure¬ 
ment  fusion  case,  the  cross-terms  went  to  zero  because  the  error  and 


noise  were  zero-mean  and  uncorrelated.  In  the  track  file  fusion  case,  the 
errors  still  have  zero-mean,  but  they  are  most  likely  correlated.  If  the 
sensors  initialize  their  estimates  with  the  same  initial  value  or  if  the  same 
process  model  is  assumed,  then  the  sensor  estimates  will  be  correlated 


and  the  cross-terms  are  nonzero.  Let 

P|2(k)  =  E[€‘j(k/n)  -r  T(k/m)l, 

(64) 

P21 W  =  =  E  (€‘2(k/m)  TT(k/n)], 

(65) 

Pjj(k)  =  E  i€j(k/n)rjV/n)], 

(66) 

and 

P22(k)  =  E  [€'2(k/m) 

(«7) 

then  our  requirement  for  a  minimum  error  estimate  is 


Pj2(k)  -  Pjj(k)  CT  +  K  (P^jCk)  -  P2j(k)  CT 
-CPj2(k)  +C  Pjj(k)  cT]  =  [0] 


and  the  optimal  gain  is 


Thus,  if  two  sensors  can  supply  Altered  estimates  of  the  target  state 
vector,  track  file  fusion  is  a  two-step  process.  Update  each  sensor’s  esti¬ 
mate  and  error  covariance  to  a  common  time  and  combine  them  with  the 
composite  gain  as  shown  in  Figure  5.  From  equation  49  and  our  expres¬ 
sions  for  Ki  and  K2.  the  track  file  fusion  algorithm  is 


Unfortunately,  accurate  calculation  of  the  gain  K  requires  that  we  com¬ 
pute  the  cross-covariance  term  P12  (h).  If  we  initialize  the  cross-covari¬ 
ance  term  based  on  our  initial  state  estimates,  then  we  can  update  the 
term  each  time  we  perform  a  track  file  fusion  update.  Recall 

PjjCk)  =  Ei€‘j(k/n)  TTck/m)]  (72) 

but  from  the  measurement  fusion  derivation  (equation  33),  we  know  that 
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Figure  5.  Track  File  Fusion  Solution 


T^(k/n)  =  (k-n)  T^(n/n)  -  G  ^  “^^Cn-l)  (73) 

and 

T^Ck/m)  =  4>2(k-m)T^(m/m)  -  G^'w^Cm-l),  (74) 

Therefore,  omitting  the  state  transition  arguments  for  brevity,  we  have 


Pj^Ck)  =  E  {[^^T^(n/n)  -  G^  "^^(n-l)] 
[^^'^(m/m)  -  ■^^'^(m-l)  G^^  ]} 


or 

Pj2 (k)  ^ j  •  E  le  ^ (n/n)  “I'J (m/m)]  • 

+  G^  E  [w  j(n-l)“w2'^(m-l)]  G^'^ 

_^T  T 

-  Gj  E  [  Wj(n-l)  e  2  (*n/nni)] 

-  <I>^  E  [6  ^(n/n)  W2^(m-1)]  G  ^ . 

The  cross-terms  are  zero,  and  if  we  let 

(k-1)  =  E \t^  (n/n)  T2T(m/m) ]  (77) 

and 

Oj^^k-l)  =  Efw  ^(n-l)"wj(m-l)]  (78) 

then  the  predicted  cross-covariance,  equation  76  becomes 

Pl2(k)  = 

The  error  covariance  based  on  the  predictions  of  e  i  and  is  (see 
measurement  fusion  equation  40) 

Pj2'(k-1)  =  e{|(I-KjC  jj-TjCn/n-l)  +  K  jtrj(n)| 

.  Ifj  (m/m-l)  (I-KjC^)^  +  ^2^(m) 

where  Kj,  Cj,  and"^i  are  the  ith  sensor’s  gain,  state-measurement 
transformation,  and  measurement  noise  vector,  respectively.  Since  the 
measurement  noises  are  uncorrelated 

Pj2(k-1)  =  (I-K^C^)  E  ^j(n/n-l)  TT(m/m-l)  ]  (I-K^C^)'^ 

(81) 

and  if  the  time  update  intervals  are  synchronized,  so  that  n=m=k-l,  then 
from  equation  72, 

P^2  (k-1)  =  E  (n/n-1)  "TJ (m/m-l)]  (82) 

and  we  can  recursively  update  the  cross-error  covariance  matrix  with 
equations  79  and  81 


and 

Pj2'(k-1)  =  (I-K  iCj)Pj2(k-l)  (I-K  (84) 

or  in  one  recursive  equation  (substitute  equation  84  into  equation  83) 


Pl2(k)  =  <l'i  (I-KjCj)  Pj2(k-1)  (I-K2C2)T 

+  G,Q22(k-l)G2T 


(85) 


With  the  above  expression  for  Pi 2  (k)  and  the  track  file  fusion  equa¬ 
tions  70  and  71,  we  can  correctly  fuse  to  track  files.  Unfortunately,  be¬ 
sides  the  computational  burden  of  computing  Pi2  (k).  the  algorithm  re¬ 
quires  that  each  sensor  pass  not  only  its  error  covariance  matrix,  Pii(k) 
or  P22  (k).  but  also  its  state  transition  matrix,  Kalman  gain  matrix,  and 
process  noise  statistics.  This  additional  data  flow  and  the  extra  computa¬ 
tional  burden  make  the  cross-covariance  term  extremely  undesirable.  If 
we  could  show  that  P12  (k)’s  contribution  is  negligible,  then  the  track  file 
fusion  algorithm  (equations  70,  71,  and  85)  becomes 


(86) 


(87) 


which  is  a  much  more  manageable  fusion  algorithm. 


■$tk)  =‘^j(k/n)  +  K  ^ 

K  =  |Pjj(k)  c”^)  .  |Pj2(k)  +  C  P,,(k)  c'^J 


2.3  Significance  of  P12  (k)  Term 

To  determine  the  significance  of  the  P12  (k)  m  track  file  fusion,  we 
created  a  simulation  that  produced  estimates  with  and  without  the  cross- 
covariance  term  P12  (k).  Figure  6  summarizes  our  simulation.  We  simu¬ 
lated  a  system  that  included  two  independent  sensors  measuring  inde¬ 
pendent  Cartesian  position  components  (that  is,  both  sensors  are  located 
at  the  origin  and  they  independently  measure  the  position  components  x 
and  y) .  Within  a  given  sensor,  the  x  and  y  measurements  are  independ¬ 
ent  of  each  other.  We  realize  that  this  sensor  suite  is  not  representative  of 
most  applications,  but  we  were  mainly  concerned  with  testing  the  signifi¬ 
cance  of  the  term  P12  (k).  We  wanted  as  simple  a  simulation  as  possible 
that  would  still  exercise  the  problem.  We  feel  that  the  simulation  exer¬ 
cises  the  Pi 2  (k)  term  better  than  most  applications.  Most  applications 
have  sensor  suites  consisting  of  dissimilar  sensors  (e.g.,  radar,  IR,  etc.), 
and  their  filtered  outputs  should  be  less  correlated  than  a  sensor  suite 
containing  very  similar  sensors.  We  initialized  both  sensors’  estimates  to 
the  mean  value  of  the  initial  state  and  gave  them  each  the  same  process 
noise  model;  we  believe  that  unless  the  filters  use  the  same  measure¬ 
ments,  this  situation  is  as  correlated  as  any  application.  If  the  simulation 
shows  that  the  contribution  of  the  P12  (k)  is  negligible,  then  we  believe 
the  term  will  be  negligible  in  most  applications. 

The  simulation  generated  one  target  at  a  random  position  with  a  ran¬ 
dom  velocity.  The  initial  position  and  velocity  vectors  were  Rayleigh  dis¬ 
tributed  in  magnitude  and  uniformly  distributed  in  direction.  The  simula¬ 
tion  initialized  the  individual  sensor  filters  (call  them  filter  1  and  filter  2) 
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PROBLEM  GEOMETRY: 


SIMULATION: 

■  Two  sensors 

■  Measure  [  X  Y  independently  with 
different  measurement  noises 

■  Estimate  [  X  X  Y  Y  for  both  sensors 

■  Fuse  estimates  with  and  without 
correlation  term 

■  Both  filters  Initialized  to  statistical 
mean  of  target  position  and  velocity 
vectors 


■  Target  appears  randomly 

•  Position  -  Velocity  vectors  are  Rayleigh  distributed  In  magnitude  and  uniformly 
distributed  in  direction 

•  it  continues  at  constant  velocity/constant  heading 


Figure  6.  Simulation  Summary 


to  the  mean  values  of  the  initial  target  state  vector.  It  implemented  a 
four-state  position-velocity  filter  in  each  sensor.  The  only  difference  be¬ 
tween  filter  1  and  filter  2  was  that  their  input  measurements  were  un¬ 
correlated  and  had  different  statistics. 

Once  initialized,  the  target  continued  moving  with  a  constant  velocity 
vector  (magnitude  and  direction)  for  the  duration  of  the  run.  White  pro¬ 
cess  noise  was  added  to  the  position  state,  and  both  sensor  filters  used 
the  same  process  noise  model.  The  two  sensor  estimates  were  fused  with 
both  track  file  fusion  techniques  (with  and  without  the  V\2  (k)  term). 
The  results  from  various  runs  are  shown  in  Figures  7,  8,  9,  and  10.  The 
plots  are  root  mean  squared  (RMS)  error  values  from  100-point  Monte 
Carlo  runs.  The  results  show  that  even  when  the  process  noise  is  high,  the 
^12  (^)  contributes  very  little.  On  most  of  the  scales  it  is  impossible  to 
distinguish  between  fusion  with  Pi2  (k)  and  fusion  without  P12  (k).  Our 
conclusion  is  that  the  P12  (k)  contribution  in  track  file  fusion  is  not  worth 
the  extra  processing  required  to  compute  the  additional  term.  This  is  not 
to  say  that  the  term  is  insignificant  in  the  association/correlation  problem. 
Further  investigation  is  required  to  determine  whether  exclusion  of  the 
cross-correlation  term  from  the  data  association  problem  adversely  af¬ 
fects  tracking  response.  We  are  saying  that  once  the  association  is  com¬ 
plete,  the  effect  of  P12  (k)  on  the  optimal  track  file  fiision  gain  is  mini¬ 
mal,  and  if  the  term  was  omitted  during  the  association  process,  then 
there  is  no  reason  to  include  it  in  the  filtering  update  algorithm. 

There  is  one  drawback  to  the  fusion  technique  discussed  above.  Look 
back  at  equations  for  track  file  fusion  (#70  and  #71).  Note  that  the  form 
is  nonrecursive;  it  depends  on  the  availability  of  data  from  both  sensors  at 
each  iteration.  If  data  is  unavailable  from  sensor  #2,  then  the  gain  ap¬ 
proaches  zero  as  P22  (ic)  grows  and  the  first  sensor’s  estimate  becomes 
the  fused  estimate.  Alternatively,  if  data  is  unavailable  from  sensor  #1, 
the  gain  approaches  one  and  the  second  sensor’s  estimate  becomes  the 
fused  estimate.  Even  though  we  may  have  been  fusing  data  for  a  period 
of  time  before  the  inputs  from  one  sensor  become  unavailable,  that  fused 
information  is  lost  with  the  track  file  fusion  form  we  have  derived.  Fur¬ 
ther  study  is  required  to  determine  whether  a  recursive  track  file  fusion 
form  should  be  employed  or  whether  the  current  form  is  adequate. 

In  summary,  if  the  measurements  from  each  sensor  are  available  and 
the  data  transfer  bandwidth  is  acceptable,  then  the  measurement  fusion 
algorithm  offers  the  best  response.  If,  however,  track  files  must  be  fused, 
then  the  track  file  fijsion  method  without  the  cross-correlation  term  of¬ 
fers  excellent  performance.  As  a  final  note,  we  would  like  to  establish 
that  the  curves  presented  in  Figures  7,  8,  9.  and  10  fail  to  illustrate  the 
full  benefits  of  fusion.  Because  we  were  looking  for  maximum  correlation 
between  sensors,  the  two  sensors  we  simulated  measure  very  similar  pa¬ 
rameters.  The  best  fusion  performance  is  achieved  when  each  sensor  in 
the  multisensor  system  measures  a  different  dimension  of  the  target  state 
(e.g.,  radar  measures  good  range/range  rate,  and  infrared  measures  good 
angle) . 
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Figure  7.  High  Measurement  Noise,  Low  Process  Noise 
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Figure  9.  High  Measurement  Noise,  Medium  Process  Noise 
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Figure  8.  High  Measurement  Noise,  Low  Process  Noise 
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Figure  10.  High  Process  Noise,  Low  Measurement  Noise 
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TRACK  INITIALIZATION  AS  AN  INTEGER  PROGRAM 
WITH  POSSIBLE  TARGET  DETECTIONS 


Scott  L.  Godfrey 


FMC  Corporation 
East  Coast  Engineering  Office 
1  Danube  Drive 
King  George,  VA  22485 


The  paper  describes  an  algorithm  for  solving  multitarget  tracking  problems  in  the 
presence  of  false  alarms  and  missed  target  detections.  The  algorithm  is  an  extension  ot 
an  algorithm  found  in  the  literature  which  did  not  consider  tracking  environments  where 
the  probability  of  detection  was  less  than  one.  The  algorithm  uses  a  Kahnan  hlter  to 
generate  a  set  of  candidate  target  tracks  from  received  measurements.  Infinitely  erroneous 
measurements  called  auxiUaxy  measurements  are  used  to  accomodate  false  alarms  and 
missed  detections.  A  method  is  described  to  estimate  a  bound  on  the  number  of  targets 
present  (nr)  in  the  surveillance  area  from  the  number  of  measurements  received  a,nd  a 
priori  models  for  the  number  of  targets  present,  the  number  of  false  alariM,  and  the 
probability  of  detection.  Integer  0-1  programming  is  used  to  pick  the  ny  best  tracks 
from  the  set  of  candidate  tracks.  Some  simulation  results  are  given  to  demonstrate  the 
operation  of  the  algorithm. 
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I.  INTRODUCTION 


Simply  stated,  multitarget  tracking  is  tbe  de¬ 
termination  of  target  trajectory  estimates  or  tracks 
from  measurement  data  originating  from  the  targets. 
The  trajectory  estimates  consist  of  parameters  such 
as  position,  velocity,  identity,  etc.,  and  the  measure¬ 
ments  can  consist  of  target  positions  and  velocities  as 
well  as  other  quantities.  Once  the  measurements  have 
been  received,  the  problem  is  not  so  much  in  updating 
the  individual  tracks  with  the  received  measurements 
a.s  it  is  in  deciding  with  which  tracks  the  measure¬ 
ments  are  associated.  Two  complicating  factors  are 
that  some  of  the  received  measurements  can  be  false 
alarms,  and  the  probability  of  detecting  a  target  can 
be  less  than  one.  Tliis  measurement-to- track  correla¬ 
tion  problem  is  the  primary  difficulty  in  multitarget 
tracking. 

Much  has  been  written  about  the  multitarget 
tracking  problem,  and  most  of  it  can  be  found  in 
the  survey  texts  by  Bar-Shalom  [ref.  2]  and  Black¬ 
man  [ref.  3].  The  methbds  used  in  addressing  the 
mecisurement-to-track  correlation  problem  have  gen¬ 
erally  involved  the  use  of  Bayesian,  maximum  like¬ 
lihood,  and  maximum  a  posteriori  estimation  tech¬ 
niques.  Using  the  Bayesian  approach  to  track  estima¬ 
tion,  a  target  track  is  updated  using  a  weighted  sum  of 
measurements  which  are  close  enough  in  a  statistical 
sense  to  the  target.  The  weight  assigned  to  a  mea¬ 
surement  is  the  probability  (likelihood)  that  the  cor¬ 
relation  hypothesis,  “the  measurement  corresponds  to 
the  track,”  is  correct.  Bar-Shalom’s  Joint  Probaba- 
bilistic  Data  Association  Filter  (JPDAF)  [ref.  1]  and 
Reid’s  multiple-hypothesis  filter  [ref.  6]  are  examples 
of  algorithms  which  use  the  Bayesian  approach. 


In  contrast,  algorithms  using  the  maximum  like¬ 
lihood  (ML)  estimation  method  or  the  maximum  a 
posteriori  (MAP)  estimation  method  update  a  track 
using  the  single  measurement  which  is  statistically 
closest  to  the  target,  or  equivalently,  they  choose  the 
correlation  hypothesis  which  is  most  likely.  The  dif¬ 
ference  between  the  two  techniques,  ML  and  MAP, 
is  that  the  MAP  technique  assumes  prior  knowledge 
about  the  likelihood  of  the  different  hypotheses  but 
the  ML  method  does  not.  The  ML  and  MAP  estima¬ 
tion  methods  are  similar  in  that  each  makes  a  “hard” 
decision  as  to  which  measurement  is  correct.  On  the 
other  hand,  the  Bayesian  estimation  makes  a  “soft” 
decision  where  measurements  with  large  likelihoods 
dominate  the  multitarget  tracking  solution,  but  mea¬ 
surements  with  small  likelihoods  stiU  have  some  in¬ 
fluence  on  the  solution. 

Morefield’s  integer  programming  algorithm 
[ref.  5]  is  an  example  of  the  ML/MAP  estimation 
methods.  Morefield  casts  the  multitarget  tracking 
problem  in  the  framework  of  pattern  recognition.  He 
uses  a  Kalman  filter  to  generate  a  set  of  candidate 
tracks  from  received  measurements.  Each  of  the  can¬ 
didate  tracks  has  an  acceptable  goodness-of-fit  to  an 
assumed  dynamic  model  and  is  considered  not  too  un¬ 
likely  to  be  of  an  actual  target.  Integer  programming 
is  used  to  select  the  best  set  of  tracks  from  the  set  of 
candidate  tracks  so  that  each  received  measurement 
is  used  in  the  computation  of  only  one  track. 

Morefield  allowed  for  the  possibility  that  re¬ 
ceived  measurements  may  be  false  alarms.  However, 
he  did  not  consider  tracking  environments  where  the 
probability  of  detecting  a  target  is  less  than  one.  This 
paper  presents  an  extension  to  Morefield’s  tracking 
method  which  allows  for  a  nonunity  probability  of 
detection.  The  algorithm  presented  is  a  batch  algo- 
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ritlim,  and  due  to  the  rapidly  growing  computational 
burden  associated  with  it  as  the  number  of  sensor 
scans  increases,  its  main  application  is  to  track  initi¬ 
ation  where  the  number  of  sensor  scans  is  relatively 
small. 

The  paper  is  organized  as  follows.  In  section  II, 
the  dynamic  target  model  and  the  Kalman  filter 
equations  are  presented  briefly.  Morefield’s  track¬ 
ing  method  is  presented  in  section  III.  The  auxiliary 
measurement  idea  is  presented  in  section  IV  with  a 
discussion  of  how  Morefield’s  algorithm  can  be  mod¬ 
ified  to  use  auxiliary  measurements  to  accomodate 
both  missed  detections  and  false  alarms.  In  section 
V,  a  method  for  estimating  a  bound  (nj)  on  the  num¬ 
ber  of  targets  in  the  surveillance  area  is  presented. 
The  algorithm  picks  the  ny  best  tracks  from  the  set 
of  candidate  tracks  as  shown  in  the  simulation  results 
presented  in  section  VI.  Conclusions  are  given  in 
section  VII. 

II.  THE  DYNAMIC  TARGET  MODEL 
AND  THE  KALMAN  FILTER 

For  the  development,  it  is  assumed  that  target 
behavior  is  described  by  the  linear  stochastic  system 

®*!  +  l  =  (1) 

Zk  =  Hxk  +  Vk,  (2) 

where  is  the  nxl  state  vector  at  time  %  and  con¬ 
tains  the  target  parameters  to  be  estimated; 
is  the  transition  matrix  which  describes  how  the  state 
of  the  system  evolves  with  time,  and  Wk  is  zero-mean 
Gaussian  process  noise  with  covariance  The  mxl 
measurement  Z)g  is  of  the  state  and  H  is  the  mea¬ 
surement  matrix  which  describes  how  the  state  and 
measurement  vectors  are  related.  The  error  in  the 
measurement  is  which  is  zero-mean,  Gaussian,  and 
has  covariance  Rk’  Also  it  is  assumed  that  the  initial 
state  of  the  system  xq  is  a  Gaussian  random  vector 
with  mean  xq  and  covariance  Fq* 

If  the  correspondence  between  received  mea¬ 
surements  and  tracked  targets  is  known,  then  trajec¬ 
tory  estimates  can  be  generated  using  the  weU-known 
Kalman  filter  equations: 


Prediction: 

^k+ljk  =  (3) 

Sfc+l/J.  =  ^k+l,k'^klk^1+l,k  +  Qk-  (4) 

Correction: 

~  ^klk-l  d"  (5) 

^k  ~  ~~  R^kfk—li  (®) 

=  (7) 


Kalman  Gain: 

Kk  =  'Sk/k-iH'^V-\  (8) 

Vk  =  HJ:k/k-iH^  +  Rk-  (9) 

Initialization: 

£o/-i=®o,  (10) 

So/_i  =  Po.  (11) 


The  subscript  m/n  denotes  a  filter  quantity  at  time 
tm  given  measurements  received  up  to  time  Thus 
Xfc/jt  denotes  the  estimate  at  time  4  given  measure¬ 
ments  received  up  to  time  The  quantity  is 
the  covariance  of  the  estimation  error  associated  with 

The  measurement  residual  or  innovation  Zk  is 
the  difference  between  the  measurement  z^  and  the 
predicted  measurement  li^as  a  covariance 

14. 

III.  THE  FORMATION  OF  TARGET  TRACKS: 
MOREFIELD’S  ALGORITHM 

Morefield  uses  a  multiple  hypothesis  approach 
to  form  tracks  from  received  measurements.  Each 
hypothesis  is  the  event  that  a  particular  set  of  mea¬ 
surements  originated  from  a  single  target  and  can 
therefore  be  used  to  compute  a  target  track.  Let 
z{i)  represent  the  measurements  received  on  scan  i 
and  let  Ui  be  the  number  of  measurements  received 
on  scan  i.  It  is  assumed  that  all  data  received  on  the 
same  scan  are  received  at  the  same  time.  Also,  let 
=  {2:(1),...,2:(A:)}  represent  the  collection  of  data, 
received  through  scan  and  —  {z^,^ ,Zjy^}  be 
a  set  of  measurements,  one  chosen  from  each  sensor 
scan.  The  subscript  Ui  serves  to  indicate  which  of  the 
measurements  received  on  scan  i  is  included  in  the 
set. 

From  the  set  of  measurements  a  target 

track  can  be  generated  using  the  Kalman  filter.  A 
chi-square  gating  criterion  must  be  satisfied  for  each 
measurement  in  before  can  be  considered  to 
form  a  candidate  target  track: 


<  7> 

(12) 

where  7  is  such  that 

V 

II 

P 

(13) 

and  a  is  small.  Inequality  (12)  describes  an  m- 
dimensional  hyper  ellipsoid,  the  interior  of  which  de¬ 
fines  the  surveillance  region  at  time  ti  in  the  measure¬ 
ment  space.  In  addition  to  the  requirement  in  (12), 
Morefield  requires  that  the  entire  track  formed  satisfy 
a  quality  constraint  so  that 

F(Z^’^  forms  a  false  track)  <  a'  (14) 
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for  a  given  a'.  Let  Q  he  the  set  of  sequences  of  mea¬ 
surements  which  are  not  too  unlikely  to  form  tracks 
of  actual  targets. 

Denote  r  =  as  a  feasible  partition  of 

the  measurements  where  nj  is  the  number  of  targets 
present  in  the  surveillance  area.  A  feasible  partition 
has  the  following  properties: 

r  C  0;  (15) 

n  ji 

Z>‘  =  \J  (16) 

and 

^k,k  pi  zk,ij  ^  0^  if  i.  ^  i.  (f 7) 

The  sequence  contains  measurements  which  can- 
not  be  associated  with  a  track  for  the  given  partition. 


The  tracks  considered  best  are  those  associated 
with  the  most  likely  partition  which  in  this  case  is  the 
one  with  the  maximum  posterior  probability  p{t\Z^), 
Using  Bayes’  rule 


p(rl^‘) 


p(Z*|r)p(r) 
p{Z'^)  ’ 


(18) 


where,  p(r)  is  the  a  priori  probability  that  a  particu¬ 
lar  partition  is  correct.  Ignoring  the  normalizing  fac¬ 
tor  p{Z^)^  the  tracks  considered  best  are  associated 
with  the  partition  which  maximizes  p{Z^\T)p{r). 

As  stated,  the  optimal  partition  is  the  maximum 
o  posteriori  estimate.  However,  if  the  prior  proba¬ 
bility  density  function  (pdf)  p(r)  is  uniform  over  aU 
possible  partitions,  i.e.  there  is  no  prior  information, 
then  the  optimal  partition  maximizes  the  likelihood 
function  p{Z^\t)  and  provides  a  maximum  likelihood 
estimate  of  the  target  tracks. 

The  problem  of  maximizing  (18)  can  be  trans¬ 
formed  into  a  more  convenient  minimization  problem 
by  using  the  negative  log  of  p(Z*^|r)p(r),  the  cost  of 
the  partition,  as  a  measure  of  the  likelihood  of  the 
partition.  Assuming  tracks  are  independent 


n  j- 

p(^*|r)  =  np(Z*’'i|r), 


(19) 


and  the  negative  log-likelihood  function  is  the  cost 

ri'p 

J  =  -ln[p(r)]  -  ^ln[p(Z*’'-’  |r)].  (20) 

i=0 

As  shown  in  [ref.  5], 

ri'p  nj’ 

-£ln[p(^*'-'i|r)]  =  (21) 

j-0 


where 


1 

=  ^  -  [ln|14,;|  -f  5^147^5^;  -f-  mln27rj  - 
i—l 

(22) 


and  Yi>.  is  the  volume  of  the  surveillance  region  de¬ 
fined  by  inequality  (12), 


(7^)^ 

r(f  +  i)  ’ 


(23) 


and  r(-)  is  the  gamma  function.  Substituting  (23) 
into  (22)  simplifies  to 


L2  2 


(24) 


The  selection  of  an  expression  for  p(r)  is  some¬ 
what  subjective.  Morefield  suggests  several  possibil¬ 
ities.  However,  since  our  ultimate  goal  is  to  initi¬ 
ate  target  tracks  in  the  presence  of  false  alarms  and 
missed  target  detections,  we  would  like  p(r)  to  be 
such  that  a  track  formed  with  a  large  number  of  mea¬ 
surements  has  a  larger  likelihood  than  a  track  formed 
with  a  small  number  of  measurements.  For  this  ap¬ 
plication,  the  form  of  the  prior  probability  is 

=  (25) 

«=1 

where  ni.  is  the  number  of  measurements  in  and 

Cn  is  the  normalization  constant 

Cn  =  E/.  (26) 

1=0 


If  the  cost  contribution  in  (21)  is  ignored,  the  form 
of  p{t)  makes  it  more  desirable  to  have  in  a  partition 
one  track  with  k  measurements  rather  than  two  tracks 
with  ki  and  k2  measurements  where  ki  k2  =  k. 
Combining  the  expression  in  (25)  with  the  expression 
in  (24),  yields  the  cost  associated  with 


=  In  Cn  —  + 


(?+')]■ 


(27) 


Morefield ’s  solution  to  the  problem  is  as  follows. 
Assume  that  there  are  L  elements  in  the  set  Q  of  fea¬ 
sible  measurement  sequences.  Let  be  a  binary 
measurement  indicator  vector  of  dimension  iVmeasj 
the  total  number  of  measurements  received.  Ones  in¬ 
dicate  which  measurements  belong  to  and  all 

other  elements  are  zeros.  Then,  an  AnieasxL  binary 
matrix  A  can  be  used  to  represent  Q.  Each  column  of 
A  is  a  measurement  indicator  vector  corresponding  to 
one  of  the  sequences  in  0.  The  costs  associated  with 
each  column  of  A  can  be  represented  in  a  IxL  vec¬ 
tor  Ct;.  Using  an  Lxl  binary  partition  indicator  vector 
p  to  indicate  which  columns  qf  A  are  included  in  a 
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particular  partition  r,  the  problem  of  finding  the  op¬ 
timal  partition  can  be  equivalently  expressed  as  the 
0-1  integer  programming  problem: 

mincv  •  p  (28) 

p 

subject  to  the  inequality  constraint 

A-p<l,  (29) 

where  1  is  a  vector  of  ones.  Equation  (28)  and  in¬ 
equality  (29)  describe  a  set  packing  problem  [ref.  4]. 
Note  that  the  measurements  not  used  in  the  forma¬ 
tion  of  tracks  as  indicated  by  the  optimal  partition  are 
the  false  alarms.  If  there  are  no  false  alarms,  then  the 
constraint  in  (29)  becomes 

A  •  /9  =  1  (30) 

requiring  each  measurement  to  be  assigned  to  a  track, 

a  set  partitioning  problem  [ref.  4]. 


2  3  4 

X  POSITION  (MILES) 


Figure  1.  Three  track  formation  hypotheses,  (a)  No 
targets  present,  (b)  Two  targets  present,  (c)  Three 
targets  present. 


An  illustration  of  how  tracks  might  be  formed 
from  measurements  is  shown  in  Figure  1.  Like  sym¬ 
bols  are  used  to  represent  measurements  received  on 
the  same  sensor  scan.  Shown  in  Figure  i(a)  are  mea¬ 
surements  received  from  three  sensor  scans.  Figure 


1(a)  also  represents  the  hypothesis  that  there  are  no 
targets  present;  all  measurements  are  false  alarms 
{A  '  p  —  0).  Figure  1(b)  depicts  the  hypothesis  that 
two  targets  are  present,  and  there  are  three  false 
alarms  (three  zeros  in  A  -  p).  Figure  1(c)  depicts  the 
hypothesis  that  three  targets  are  present  and  there 
are  zero  false  alarms  (A  •  p  =  1). 

IV.  THE  FORMATION  OF  TARGET  TRACKS: 
AUXILIARY  MEASUREMENTS 


If  the  probability  of  detection  is  less  than  one, 
then  the  number  of  hypotheses  which  must  be  con¬ 
sidered  in  forming  tracks  is  infinite.  For  example,  in 
Figure  1,  a  large  number  of  targets  could  be  present  in 
the  surveillance  region,  but  the  sensors  fail  to  detect 
them.  Although,  in  most  situations,  this  is  unlikely. 
The  approach  presented  in  this  section  is  to  account 
for  the  missed  detections  by  completing  the  surveil¬ 
lance  picture  with  artificial  measurements  called  aux¬ 
iliary  measurements.  When  a  target  fails  to  be  de¬ 
tected,  an  auxiliary  measurement  is  used  to  update 
the  target’s  track.  The  auxiliary  measurements  are 
infinitely  erroneous  and  therefore  have  no  influence 
on  tracks  formed  with  them.  Using  this  approach, 
the  multitarget  tracking  problem  can  be  solved  as  a 
set  partitioning  problem  using  efficient  integer  pro¬ 
gramming  techniques. 

In  the  operation  of  a  track  processing  system, 
the  Kalman  filters  used  for  state  estimation  extrapo¬ 
late  corrected  target  estimates  to  predict  where  tar¬ 
gets  are  likely  to  be  when  measurements  are  taken, 
and  the  predicted  estimates  are  updated  with  the  re¬ 
ceived  measurements.  If  no  measurement  is  received 
from  a  target  (a  missed  detection),  the  track  proces¬ 
sor  skips  the  correction  phase  of  the  Kalman  filter 
and  predicts  where  the  target  is  likely  to  be  the  next 
time  measurements  are  taken.  Eventually,  either  a 
new  measurement  is  received  and  the  target’s  track  is 
updated,  or  the  track  becomes  stale,  and  it  is  termi¬ 
nated. 

Skipping  the  correction  phase  of  the  Kalman  fil¬ 
ter  is  equivalent  to  setting  the  Kalman  gain  equal  to 
zero  and  updating  the  predicted  estimate  as  if  a  mea¬ 
surement  had  been  received.  Define  an  auxiliary  mea¬ 
surement  as  a  measurement  which  contains  no  infor¬ 
mation,  i.e.,  the  covariance  of  the  measurement  error 
associated  with  it  is 

R  =  diag  [  oo, . . . ,  (X)  ]  .  (31) 

This  makes  the  covariance  of  the  measurement  resid¬ 
ual 

.  Vt  =  +  Rk  (32) 

infinite  and  the  Kalman  gain  zero.  The  auxiliary 
measurement  is  infinitely  erroneous  so  that  updat¬ 
ing  a  target  track  with  it  is  equivalent  to  declaring 
a  missed  detection  of  the  target.  Naturally,  since  an 
auxiliary  measurement  acts  as  a  missed  detection,  its 
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Figure  2.  Track  formation  for  <1. 
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presence  is  not  included  in  the  cost  computation  of 
equation  (27). 

The  auxiliary  measurement  idea  is  a  de¬ 
vice  which  fits  nicely  into  the  appealing  predic¬ 
tion/correction  structure  of  the  Kalman  filter.  With 
it,  there  is  always  a  measurement  with  which  to  up¬ 
date  a  track.  In  the  present  approach  to  track  initia¬ 
tion,  aiixihary  measurements  are  used  to  accomodate 
both  missed  detections  and  false  alarms.  The  under¬ 
lying  philosophies  are  that  first,  an  auxiliary  mea¬ 
surement  wiU  be  used  to  fill  in  a  track  when  a  missed 
detection  occurs,  and  second,  false  alarms  wiU  be  iso¬ 
lated  in  tracks  consisting  predominately  of  auxiliary 
measurements.  Heuristically,  the  first  point  follows 
because  an  auxifiary  measurement  has  the  same  effect 
on  a  track  as  a  missed  detection.  The  second  point 
follows  because  to  the  track  initiation  procedure,  a 
false  alarm  is  a  detection  of  a  target  which  reports  on 
only  a  single  scan.  Auxiliary  measurements  are  used 
to  account  for  the  missed  detections  of  the  “target” 
on  the  other  sensor  scans. 

To  see  how  the  track  initiation  problem  can  be 
transformed  to  a  set  partitioning  problem,  assume 
that  the  supply  of  auxihary  measurements  is  infi¬ 
nite  and  that  the  constraint  in  inequality  (14)  is  re¬ 
laxed.  In  this  case,  every  received  measurement  can 
be  placed  in  a  track  computed  from  itself  and  a  col¬ 
lection  of  auxihary  measurements.  The  tracks  are 
feasible  because  the  erroneous  nature  of  an  auxihary 
measurement  ensures  that  inequahty  (12)  is  satisfied. 
Although  these  tracks  are  trivial,  their  existence  en¬ 


sures  that  there  is  at  least  one  partition  of  the  mea¬ 
surements  which  assigns  every  measurement  to  ex¬ 
actly  one  track. 


Table  1. 

A  Measurement  Partition  for  Data  in  Figure  2. 


TRACK 

MEASUREMENT 

MEASUREMENT 

NUMBER 

NUMBER 

SCAN 

(a:,  y) 

2 

1 

(7.8088,  8.4710) 

1 

4 

2 

(19.2519,  21.4220) 

10 

3 

(29.3275,  31.0576) 

1 

1 

(51.1203, -0.1941) 

2 

8 

2 

(39.2674,  10.5393) 

9 

3 

(28.4304,  17.49249) 

1 

AM 

3 

5 

2 

(64.6668,  50.1273) 

11 

3 

(60.6907,  61.1080) 

3 

1 

(-9.6299,  35.4111) 

4 

7 

2 

(16.3081,  81.7034) 

3 

AM 

1 

AM 

5 

6 

2 

(98.6156,  72.1212) 

3 

AM 

As  an  example  of  how  feasible  tracks  can  be 
formed  using  auxihary  measurements,  consider  the 
situation  in  Figure  2.  The  figure  shows  measurements 
from  three  targets  received  over  three  sensor  scans 
with  a  probability  of  detection  of  0.75.  The  actual 
target  tracks  are  indicated  by  the  dotted  fines.  False 
alarms  are  assumed  to  be  uniformly  distributed  in 
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the  surveillance  area  where  the  number  occuring  on 
a  scan  is  modeled  as  Poisson 


P{n  FA)  = 


n! 


(33) 


where  Y  is  the  volume  of  the  surveillance  area  and  A 
is  the  mean  number  of  false  alarms  per  unit  volume 
of  surveillance  area.  Here,  XY  was  chosen  as  1.  A 
partition  of  the  measurements  showing  the  measure¬ 
ments  used  to  generate  five  candidate  target  tracks  is 
shown  in  Table  1.  The  measurement  numbers  corre¬ 
spond  to  those  in  Figure  2.  Of  course,  this  is  not  the 
only  measurement  partition. 


V.  ESTIMATING  THE  MAXIMUM 
NUMBER  OF  TARGETS  PRESENT 


In  this  section,  the  aim  is  to  compute  a  bound 
on  the  number  of  targets  in  the  surveillance  area  so 
that  a  complete  partitioning  of  the  measurements  is 
not  required.  The  bound  is  computed  based  upon 
the  number  of  measurements  received  on  each  sensor 
scan  and  a  priori  models  for  the  number  of  targets 
present  in  the  surveillance  area,  the  number  of  false 
alarms,  and  the  probability  of  detection.  The  goal  is 
to  speed  the  solution  of  the  problem  by  dropping  the 
requirement  that  the  measurements  be  partitioned, 
and  instead  select  the  ny  best  target  tracks  as  a  so¬ 
lution. 

Assume  measurement  data  are  received  for  N 
sensor  scans.  Assume  that  the  number  of  targets 
present  during  the  reception  of  measurements  is  con¬ 
stant,  i.e.  targets  neither  disappear  nor  appear  during 
the  N  sensor  scans.  It  is  required  that  nx  be  com¬ 
puted  such  that 

P(number  of  targets  >  nx\N  scans  of  data)  <  a 

(34) 

where  a  is  small.  Let  ni  represent  the  number  of 
measurements  (target  detections  and  false  alarms)  re¬ 
ceived  on  scan  2,  and  be  the  set  indicating 

the  number  of  measurements  received  on  the  N  sen¬ 
sor  scans.  Then  an  equivalent  problem  is  to  compute 
nx  such  that 

F(number  of  targets  >  <  oc.  (35) 

Let  Nx  represent  the  random  variable  for  the  number 
of  targets  present.  Then 


and  the  total  probability  theorem 


P(k\{nj}f=i) 


P{{ni}^^r\k)P(k) 


where  P(k)  is  the  a  priori  probability  that  k  tar¬ 
gets  are  present  in  the  surveillance  region.  A  Poisson 
model  is  used  for  P{k) 


P{k)  = 


^1 


(39) 


Here,  Y  is  the  volume  of  the  surveillance  area  and 
K  is  the  mean  number  of  targets  per  unit  volume  of 
surveillance.  The  mean  number  of  targets  present  is 


kY, 


The  expression  for  P{{T^j}f=,i\k)  is  needed  next. 
Since  the  number  of  targets  present  is  constant,  the 
detection  experiments  are  independent  on  each  scan. 
Therefore, 

N 

i=i 


If  Tij  <  k, 


P{nj\k)  =  ^  P[i  target  detections,  —  i  FA|fc). 

(41) 

If  Uj  >  k^ 


k 

P(nj\k)  =  target  detections,  nj  —  i  FA|fc). 

i=o 

(42) 

Equation  (42)  follows  because  there  can  be  no  more 
than  k  target  detections  if  there  are  k  targets.  Assum¬ 
ing  that  false  alarms  are  independent  of  the  number 
of  targets  present 


P{i  target  detections,  —  i  FA|/j)  = 

P{i  target  detections|fc)P(nj  -  i  FA).  (43) 


The  number  of  false  alarms  received  is  assumed  to 
obey  a  Poisson  model 


P{nj  ~iFA)  = 


{rij  -  i)l 


(44) 


and  the  probability  of  i  target  detections  is  given  by 
the  binomial  probability 


F(Nt  >  nr|{n,}f=i)  =  1  -  F(Nt  <  nr|{nj}f=i) 

(36) 

and 

Tl'p 

P{Nt  <  nT|{ni}f=i)  =  ^ P(fcl{n,}f=,)  (37) 

ik=0 

where  P{k\{nj}^_-^)  is  the  probability  that  k  targets 
are  present  given  the  received  data.  Using  Bayes’  rule 


e(»lfc)  =  (i)pji(i-w)‘''-  (45) 

Using  (43)  and  substituting  (44)  and  (45)  into  (41) 
and  (42)  yields  for  nj  <  k 

t_,(A17V-^ 

Pd)  (  ■\i  ) 

(Uj  -  l)] 

(46) 


P{nj\k)  =  YQpii(l- 
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and  for  nj  >  k 


P{nj\k)  =  ^  - 

i=o 


Pd) 


rii-i^-XY 


(rij  -  i)! 


Substituting  (46)  and  (47)  into  (40)  yields 


(47) 


^’(K}f=i|fc) 


AT  niin(*,nj) 

n  E 

j  =  l  r=0 


Pdi^  -  Pd) 


k—i 


[Uj  -  i)\ 


.(48) 


Equation  (48)  can  be  used  in  equations  (37)  and  (38) 
to  compute  nx  for  a  given  a.  Obviously,  for  any  imple¬ 
mentation  the  sum  in  the  denominator  of  (38)  must 
be  truncated. 


The  dimensions  were  miles  for  positions  and  miles 
per  hour  for  speeds.  A  diffuse  model  was  used  for  the 
initial  state  {Pq  =  diag[oo  oo  oo  oo]),  and  the 
covariance  of  the  process  noise  was 


0 

T2^2 

0 

3 

2 

0 

3 

0 

T^cr^ 

2 

2 

0 

0 

1 - 

O  1 

T2^2 

2 

0 

- 1 

<N 

b 

where  T,  the  time  between  sensor  scans,  was  chosen  as 
1  hour  and  was  chosen  as  0.25.  The  form  of  the  co- 
variance  in  equation  (50)  is  derived  by  assuming  that 
the  process  noise  is  the  system  response  to  a  continu¬ 
ous  time  white  noise  input.  Synthetic  measurements 
of  target  positions  were  generated  according  to 


Table  2. 

Bound  on  the  Number  of  Targets 
for  Several  Conditions 


(Ay) 

[kY] 

(Pd) 

CONFIDENCE 

Nx 

1 

1.0 

4.0 

0.9 

90  % 

4 

2 

1.0 

4.0 

0.7 

90  % 

5 

3 

3.0 

4.0 

0.9 

90  % 

3 

4 

1.0 

2.0 

0.9 

90  % 

4 

5 

1.0 

6.0 

0.9 

90  % 

4 

6 

5.0 

4.0 

0.9 

90  % 

2 

7 

1.0 

4.0 

0.5 

90  % 

7 

8 

1.0 

4.0 

0.9 

99  % 

5 

Examples  for  nx  are  shown  in  Table  2. 
Data  was  received  for  five  sensor  scans  where 
=  {4,5, 3, 5, 4}.  The  sum  in  the  denomina¬ 
tor  of  equation  (38)  was  truncated  at  m  =  20.  For 
the  case  shown,  the  top  row  in  the  table  represents 
nx  for  a  nominal  set  of  parameters  (mean  no.  false 
alarms  AE,  mean  no.  targets  kY^  probabiHty  of  de¬ 
tection  Prf,  and  confidence  level).  For  this  case,  as 
seen  from  rows  2,  3,  6,  and  7,  nx  is  most  sensitive  to 
the  assumed  false  alarm  rate,  and  the  probability  of 
detection. 


VI.  SIMULATION  RESULTS 


To  hmit  the  number  of  candidate  target  tracks, 
a  simple  m-contacts-out-of-A^-sensor-scans  test  was 
used  to  accomplish  the  track  quahty  test  in  inequality 
(14)  where  the  number  m  depended  upon  the  prob¬ 
ability  of  detection.  Five  constant  velocity  targets 
were  observed  over  five  sensor  scans  under  varying 
conditions.  The  targets  were  tracked  in  a  Cartesian 
(a:,y)  coordinate  frame,  and  were  assumed  to  travel 
at  constant  speed  with  process  noise  accounting  for 
deviations  from  this  behavior.  The  motion  model  was 
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where  the  measurement  error  was  Gaussian  with  co¬ 


variance 

P“diag[l.O  1.0 ]  •  (52) 


z 

o 

ijy 

O 


□  o  a 


LEGEND 
□  SCAN  1 
o  SCAN  2 
^  SCAN  3 
4-  SCAN  A 
X  SCAN  5 


20  -40  60 

X  POSITION  (MILES) 


(a) 


Figure  3.  Track  Formation  for  Scenario  1.  (a)  Re¬ 
ceived  measurements,  (b)  Initialized  tracks. 


404 


In  the  first  scenario,  the  probability  of  detection 
was  chosen  as  0.95,  the  false  alarm  rate  (AE)  as  1  per 


scan,  and  the  a  priori  mean  number  of  targets  present 
(kY)  as  3.  The  data  received  are  shown  in  Figure 
3(a),  and  the  bound  (ny)  on  the  number  of  targets 
present  was  computed  as  6  with  a  confidence  of  90%. 
To  be  a  candidate  track,  4  contacts  had  to  be  received 
over  the  5  sensor  scans  in  addition  to  having  to  satisfy 
inequality  (12)  for  7  =  9.21,  the  99%  threshold  for 
a  two  degree-of-freedom  chi-square  random  variable. 
The  tracks  initiated  by  the  procedure  are  shown  in 
Figure  3(b).  The  tracks  are  correct. 

In  the  solution  to  the  problem  only  5  tracks 
were  initiated,  but  ny  was  computed  to  be  6.  The 
missing  track  consists  entirely  of  auxiliary  measure¬ 
ments.  Tracks  consisting  of  auxiliary  measurements 
are  counted  as  candidate  tracks  in  order  to  “take 
up  the  slack”  in  the  integer  programming  problem 
if  needed. 


So. 

o'” 

o. 

>  s 


LEGEND 
□  SCAN  1 
o  SCAN  2 
^  SCAN  3 
+  SCAN  4 
X  SCAN  5 


(a) 


20  40  60  80 

X  POSITION  (MILES) 


Figure  4.  Track  Formation  for  Scenario  2.  (a)  Re¬ 
ceived  measurements,  (b)  Initialized  tracks. 


In  the  second  scenario,  all  parameters  remained 
the  same  as  in  the  first  scenario  except  that  the  prob¬ 
ability  of  detection  was  lowered  to  0.7  and  the  false 
alarm  rate  was  increased  to  3.5  per  scan.  For  this 
case,  3  contacts  had  to  be  received  over  the  5  sensor 
scans  before  a  track  could  be  considered  a  candidate 
track.  The  received  data  for  this  case  is  shown  in 
Figure  4(a).  The  bound  on  the  number  of  targets 
present  was  computed  to  be  5.  The  initiated  tracks 
are  shown  in  Figure  4(b).  Track  4  is  a  false  track,  but 
as  seen  from  the  previous  scenario,  the  other  tracks 


are  correct,  and  two  targets  were  not  detected.  Note 
that  track  2  was  initiated  using  four  measurements 
and  track  3  was  initiated  with  three  measurements 
indicating  one  and  two  missed  detections  in  the  re¬ 
spective  tracks. 


VII.  CONCLUSIONS 


A  track  initiation  algorithm  has  been  presented 
capable  of  initiating  target  tracks  in  the  presence  of 
false  alarms  and  missed  detections.  The  algorithm 
described  is  a  modification  of  an  algorithm  developed 
by  Morefield  which  addressed  multitarget  tracking  in 
the  framework  of  pattern  recognition.  Morefield  ad¬ 
dressed  false  alarms  but  did  not  address  missed  detec¬ 
tions.  It  has  been  shown  that  by  using  infinitely  erro¬ 
neous  measurements  called  auxiliary  measurements, 
the  multitarget  tracking  problem  can  be  solved  as  a 
set  partitioning  problem  using  integer  programming 
so  that  all  measurements  are  placed  in  tracks. 

A  method  was  described  to  estimate  a  bound  on 
the  unknown  number  of  targets  present  in  the  surveil¬ 
lance  area.  The  purpose  of  the  bound  was  to  improve 
the  efficiency  of  the  algorithm  so  that  the  track  initi¬ 
ation  problem  could  be  solved  as  a  set  packing  prob¬ 
lem  using  integer  programming.  Simulation  results 
were  presented  to  demonstrate  the  algorithm’s  per¬ 
formance  under  varying  conditions. 

Although  the  algorithm  presented  can  be  used 
to  solve  the  general  multi-target  tracking  problem,  its 
application  is  limited  because  of  the  rapidly  growing 
computational  burden  associate  with  it  as  the  num¬ 
ber  of  scans  of  data  increases.  Therefore,  its  main 
application  is  to  the  track  initiation  problem. 
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The  purpose  of  this  paper  is  to  describe  a  mathematical  system  that  extends 
deterministic  inferencing  mechanisms  statistically.  The  application  for  Com¬ 
mand  and  Control  is  to  faithfully  infer  knowledge  from  statistical,  as  well  as 
incomplete,  data  sources.  The  question  of  optimizing  information  storage,  while 
still  meeting  computational  requirements,  is  also  analyzed. 

The  mathematical  structures  to  be  discussed  are  an  outgrowth  of  classical 
hypothesis  testing,  but  cover  evidential,  Bayesian  and  other  reasoning  systems. 
The  viewpoint  is  that  we  wish  to  model  not  the  state  of  an  object,  but  knowl¬ 
edge  of  the  state.  As  an  example,  if  a  fair  coin  is  tossed  one  can  say  that  the 
probability  of  heads  is  0.5,  but  suppose  that  there  is  a  50%  chance  that  the  coin 
is  fair  and  a  50%  chance  that  it  is  some  unknown,  possibly  double-headed,  coin. 
Now  what  is  our  knowledge  of  the  state  of  the  coin?  Ignoring  the  question  of 
whether  the  coin  is  fair  or  not  we  arrive  at  the  following  ''conclusion'';  There 
is  a  25%  chance  that  it  is  heads,  a  25%  chance  that  it  is  tails,  and  a  50%  chanc 
that  we  are  ignorent  of  the  coins  state.  In  this  process  we  have  constructed 
a  random  variable  that  does  not  represent  the  state  of  the  coin,  but  an  infer¬ 
ence  about  the  state  of  the  coin. 

Since  Bayesian  as  well  as  evidential  reasoning  methods  are  anaylzed  on  an 
equal  footing  it  is  possible  to  determine  the  benefit  and' cost  of  choosing  one 
over  the  other.  It  is  also  possible  to  form  hybrid  schemes  that  make  desirable 
compromises.  For  example,  a  major  argument  against  evidential  reasoning  meth¬ 
ods  is  the  exponential  growth  of  memory  and  computational  resources.  The  rea¬ 
son  this  growth  occurs  is  that  there  is  no  pruning  of  hypothetical  instances. 

It  is  possible,  in  this  analysis,  to  alter  the  combination  formulas  to  perform 
automatic  pruning  with  respect  to  predetermined  rules. 
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INTRODUCTION 


The  purpose  of  this  paper  is  to  describe 
how  deterministic  inferencing  mechanisms  can 
be  adapted  to  data  sources  which  are  noisy, 
incomplete,  or  inconsistent.  While  the  view¬ 
point  of  the  approach  is  based  upon  eviden¬ 
tial  reasoning,  the  constructions  are  purely 
probabilistic.  The  focus  will  be  on  modeling 
the  knowledge ■ gained  from  the  data  sources. 
The  knowledge  will  depend  on  how  randomness, 
or  noise,  is  introduced  into  a  particular 
data  source.  We  will  also  address  problems 
that  arise  in  trying  to  meet  computational 
requirements . 


BAYESIAN  AND  RANDOM  INFERENCES 

In  Bayesian  analysis  we  are  given  a 
state  space  5  and  a  space  of  observations,  or 
measurements,  M.  We  make  a 

measurement  which  is,  in  general,  a  subset  A  of 
M  and  then  compute  a  distribution  on  the 
possible  states  of  the  system.  To  do  this  we 
need  to  have  information  on  how  the  states 
and  the  measurements  are  correlated.  This 
information  is  usually  expressed  in  terms  of 
conditional  probabilities.  That  is,  for  each 
s  e  S  there  is  a  probability  measure  P,(-)  on  M, 
i.e.  if  A  is  a  subset  of  M  then 

P,(A)  =  Prob(Als) . 


In  Bayesian  analysis  the  system 
is  described  by  an  a  priori  distribution  p  on  S . 
If  we  are  given  this  distribution  p  and  we 
have  a  mea¬ 
surement  A  c  M,  then  we  can  use  Bayes  formula 
to  compute  a  new  distribution  p' . 


/(s)  =  Prob(s|A) 


Ps{A)pis) 

EiesPt{A)pity 


Note  that  the  state  is  considered  to  be 
a  random  variable.  In  our  viewpoint,  we  will 
be  interested  in  modeling  not  the  state,  but 
our  knowledge  of  the  state  as  determined  by 
the  observation.  In  this  way  we  may  view  the 
state  as  being  deterministic,  but  unknown. 
This  allows  us  to  dispense  with 
the  a  priori  distribution.  Consider  the  classi¬ 
cal  Bayesian  scheme  described  where  for  each 
s  in  some  state  space  5  we  have  a  distribu¬ 
tion  P,(-)  on  a  space  of  observations  M,  We 
suppose  that  each  distribution  P,{-)  is  the 
distribution  of  a  random  variable  JY,  with 
values  in  M.  What  this  gives  us  is  a  random 
map  from  S  to  M,  that  is 


From  the  random  map  X  we  will  construct  a 
map  from  P{M)  to  P{S),  where  P{B)  denotes  the 
power  set  of  a  set  B.  This  random  map  will 
be  denoted  by  and  is  defined  for  A  a 
subset  of  M  by 


X-^{A)  =  {seS\X,  eA}. 

Note  that  anytime  the  conditional 
distributions  p,(-)  are  given  one  can  construct 
this  map.  The  construction  depends  on  the 
selection  of  the  random  variables  X,,  which 
is  not  unique,  but  we  shall  see  that  there 
are  properties  that  this  map  possesses  that 
are  independent  of  the  random  variables 
chosen.  We  shall  also  see  what  additional 
information  is  contained  in  the  selection. 

A  question  of  interpetation  arises  about 
the  map  X~^ .  By  constructing  the  random 
variables  X,,  we  have  not  only  modeled  the 
statistical  dependence  of  the  observations  on 
the  state,  but  the  actual  dependence,  the 
difference  being  one  of  knov/ing  the  actual 
random  variable  verses  knowing  just  its 
distribution.  The  additional  information 
allows  us  to  draw  conclusions  about  the  state 
from  an  observation.  Given  the  observation 

A  CM, 
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the  conclusion, 


seX-\A), 

represents  the  possible  states  that  could 
have  given  that  particular  observation.  In 
the  next  section  we  will  abstract  this 
notion,  but  for  now  we  will  think  of  the 
results  of  our  'inferences'  as  being  subsets 
of  5.  The  randomness  in  the  inference  arises 
from  the  inherent  random  quality  of  the 
observation . 

In  order  to  make  coherent  decisions 
based  on  the  outcome  of  our  calculations  it 
is  necessary  to  have  a  firm  probabilistic,  or 
statistical  interpretation  of  the  results. 

We  view  elements  of  P{S)  as  hypotheses  about 
the  state  of  a  system.  Let  ft  be  a  hypothesis 
and  let  A  be  an  observation,  we  compute 

Pl(ft)  =Prob(X-H^)nft7^0), 

which  reads,  the  plausibility  of  ft.  Computing  the 
plausibility  in  terms  of  the  map  X-^  we  have 
the 

PI  (ft)  =  ProbCX,  €>1  for  some  s  satisfying  ft) 

=  Prob(X,  eA\h) 

=  Prob(v4|ft). 

Note  that  comparing  the  plausibility  against 
a  threshold  is  equivalent  to  testing  the 
hypothesis  after  normalizing.  The  normalized 
version  of  the  above  computation  simply 
requires  us  to  condition  on  the  event  that  A 
is  observed.  This  amounts  to  dividing  by 
Pi  (S)  =  Prob  (>1|S)  .  What  we  then  compute  is  the 
probability  that  some  seft  would  result  in  the 
observation  A  given  that  some  s  e  S  would  re¬ 
sult  in  the  observation  A.  A  brief  example 
will  help  to  clarify  the  issues  involved. 

EXAMPLE:  THE  FAULTY  SENSOR  I 

Suppose  that  a  coin  is  not  tossed,  but 
you  are  relying  on  some  device  to  detect 
whether  it  is  heads  or  tails.  Unfortunately 
this  device  only  works  a  certain  percentage 
of  the  time.  Let's  say  that  the  probability 
of  the  device  working  correctly  is  p.  When 
it  does  work  it  correctly  reports  the  state 
of  the  coin,  and  when  it  doesn't  work  the 
output  of  the  device  is  independent  of  the 
state  of  the  coin.  We  will  suppose  that  this 
output  can  be  either  heads  or  tails  with 
equal  probability. 

If  the  device  informs  you  that  the  coin 
is  heads  what  can  we  say  about  the  state  of 
the  coin?  If  we  compute  conditional 
probabilities  we  find 

Prob  (device  says  heads]  heads)  =  (l+p)/2 
Prob(  device  says  heads]  tails)  =  (l-p)/2. 

To  use  Bayes  formula  we  have  to  make  an 
assumption  about  the  a  priori  state  of  the  coin. 
We  assume  that  the  state-  of  the  coin  is 
random  and  that  either  heads  or  tails  are 
equally  likely.  If  we  do  so,  then  using 
Bayes  formula  we  find 

Prob  (Heads)  =  (l  +  p)/2 
Prob  (Tails)  =  (l-p)/2. 

That  is,  the  state  of  the  coin  is  still 
random,  only  the  probability  distribution  has 
been  changed. 


On  the  other  hand  if  didn't  assume  that 
the  state  of  the  coin  is  random  and  if  we  use 
the  above  construction  to  model  the  random 
inference  we  find  that  we  have  a  set  valued 
random  variable  A^X-\heads)  such  that 

Proh  {A  =  {heads, tails])  =  (l-p)/2 

F r oh  {A  =  [heads])  =  p 

Froh{A  =  <l>)  =  (l-p)/2. 

If  we  condition  off  the  empty  set  we  have 

Froh  {A  —  [heads, tails])  =  (l-p)/(l+P) 

Prob  (v4  =  {ftearfs})  =  2p/(l+p)- 

In  the  first  calculation  the  state  of 
the  coin  is  random  and  in  the  second,  what  we 
infer  about  the  coin  state  is  random.  We  can 
now  easily  test  the  hypothesis  that  the  coin 
is  heads 

PI  (heads)  =1 

which  means  that  we  have  not  received  any 
information  to  decrease  the  likelihood  of ^ 
heads,  while  testing  the  hypothesis  of  tails 
results  in 

PI  (tails)  =  (1 -p)/(l +p)- 

There  are  two  things  to  note  here.  The  first 
is  that  both  methods  give  the  same  results  as 
far-  as  this  standard  statistical  test .  The 
second  is  that  by  studying  the  inference 
mechanism  involved  we  have  a  natural  language 
for  handling  the  logical  relationships 
between  hypotheses.  In  this  example,  as  it 
turns  out,  we  have  more  accurately  modeled 
the  randomness  of  the  experiment  by  the  sec¬ 
ond  approach.  Now  what  happens  when^  we  as¬ 
sume  the  existence  of  an  a  priori  distribution? 

Suppose  that  Z  is  an  S  valued  random 
variable  such  that 

Prob(Z6B)  =p{B). 

We  can  think  of  Z  as  taking  values  in  the 
singleton  sets.  To  avoid 

confusion,  define  Z  -  [Z] .  Given  Z  and  X~  ,  we 
can  logically  combine  them.  Define  a  map  Y 
from  P{M)  to  P{S)  by  . 

YiA)  =  X-HA)f)Z. 

Note  that  Y  takes  values  on  the  singleton 
sets  and  the  empty  set.  If  we  condition  y(^) 
off  the  empty  set  then  it  can  be  thought  of 
as  an  S  valued  random  variable.  If  we  then 
compute  the  distribution  of  this  random 
variable  we  arrive  at  Bayes  formula. 

p'{s)  =  Prob  (Y(>1)  =  {s}ty(i4)  ^  (?!») 

=  Prob  (y(^)  =  M) /Prob  (y(^)^</>) 

Prob({Z}n^“H^)  =  {^}) 

"  Froh  {[Z]r[X-\A)^<f>) 

Prob(Z  =  s,ZeX-M^)) 

FrohiZ  eX-^{A)) 

=  Froh  {Z  =  s,Xze  A) /Proh{Xz€  A) 

=  Prob  {s]i4) 

This  calculation  is  independent^  of  the  choice 
of  the  random  variables  X,.  This  calculation 
shows  that,  if  desired,  we  can  still  perform 
Bayesian  analysis. 

In  the  absence  of  an  a  priori  distribution 
we  could  assign  masses  to  sets  as  in 
evidential  reasoning.  For  B c 5  we  define 

m^(B)  =  Prob(X-H^)  =  ^)- 
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Note  that  this  mass  depends  explicitly  on  the 
observation,  or  evidence  A.  The  above 
equation  represents  unnormalized  mass  i.e. 
there  is  a  potential  nonzero  mass  assigned  to 
the  empty  set.  This  mass  is 

ttiaW  =  Proh(X'^(A)  =  ^} 

=  Proh.(XseA^VseS), 

where  A^  is  the  complement  of  ^  in  M.  The 
corresponding  normalized  mass  is  given 
by  conditioning  on  X~^  ^  <f>.  For  example,  the 
mass  on  B  given  is  not  empty  is 

m{B)  =  P  [X-\A)  =  B\X-^{A)i^<i>) 

Prob  =  B}  X-\A)  ^  (f>) 

Proh{X~^{A)^<f>) 
Prob(X-^(^)-B) 

(l-Prob(X-n^)  =  <^)) 


If  we  perform  an  independent  experiment 
then  this  gives  rise  to  an  independent  col¬ 
lection  of  random  variables  X,  pai^metrized 
by  S  and  with  values  in  M,  where  M  is  a  new 
space  of  observations  (perhaps  equal  to  M)  . 

By  independent,  we  mean  that  for  each 
s'  €  Sf  Xj'  is  independent  of  the  collection  o_f 
random  variables  {X,|s  e  5}  and  similar^  for  X 
and  X  reversed.  From  the  collection  X^,  we 
construct  'x~^  from  p(M)  to  P(5)  and  the_corn_ 
responding  mass  function  m-j  for  each  A  g  P{M). 
We  can  now  logically  combine  X~'^  and 

defining  the  map  Y  from  P(M)  x  P(M)  to  P{S) 
by  _  _  _ 

Y(A,A)  =  X-\A)f)X'\A) 

If  we  compute  the  corresponding  mass  function 
for  Y  we  get 

-  Proh  {Y{A,  A)  =  B) 

=  Prob(X-HA)n^“^W  =  ^) 

=  Y,  Prob{X-H^)  =  C'.  = 

Cf]D=B 

=  Y  {X~^{A)  =  C)  Proh  {ir\A)  =  D) 

Cf\D=B 

=  2  m^Qm^D) 

Cf]D^B 

This  is  the  unnormalized  version  of 
Dempster's  rule  of  combination.  To  normalize 
we  condition  Y  off  the  empty  set.  This 
amounts  to  dividing  by 

1  - 


Note  that  the  only  difference  between 
Bayesian  reasoning  and  evidential  reasoning 

is  the  type  of  inferences  that  were  combined. 
In  Bayesian  reasoning  we  combined  the 
inference  with  an  inference  derived  from 
an  a  priori  distribution.  In  evidential  reason¬ 
ing  we  combined  inferences  derived  from  two 
independent  observations. 

It  is  also  possible  to  compute  mass 
distributions  on  sets  from  the  conditional 
probabilities  found  in  the  purely  Bayesian 
approach.  If  we  assume  that  the  random 
variables  Xt  are  independent 

then  the  normalized  mass  on  a  set  B  in  S  given  a 
measurement  X  in  M  is 


rriAiB)  = 


If  we  then  combined  this  distribution  with 
the  mass  distribution  derived  from 
an  a  priori  distribution  using  Dempsters  rule  we 
would  have  Bayes  formula. 


In  the  next  section  we  will  study 
properties  of  inferences  a  little  more 
closely  and  we  shall  consider  a  variety  of 
combination  formulas  that  will  narrow  the  gap 
between  evidential  and  Bayesian  reasoning 
further. 


INFERENCE  MAPS 


We  wish  now  to  focus  our  attention  on 
X-'^ .  This  is  a  random  map  from  the  power  set 
P{M)  to  the  power  set  P{S).  The  natural 
generalization 

of  a  power  set  is  a  ktUce  .  Breifly,  a  lattice 
L  is  a  partially  ordered  set  with  two 
operations,.  A  and  v,  with  the  property  that 

aAb  ~  inf{a,b} 
aVb  —  sup{a,b} 

(sup  =  least  upper  bound  and  inf  =  greatest 
lower  bound) .  Since  the  lattices  encountered 
here  will  be'  finite,  we  can  extend  this 
operation  to  arbitrary  subsets.  For  example, 
if  L'  is  a  subset  of  L  then 

sup(L')  =  y  I 
leL' 

inf(L')  =  /\  ( 
leL' 

both  exist  in  X.  In  particular,  we  define  0 
to  be  the  siip(L)  and  <i>  to  be  the  tn/(X).  Note 
that  the  power  set,  P{S),  can  be  thought  of  as 
a  lattice  of  propositions  about 
an  element  s  e  S,  where  the  proposition  corre¬ 
sponding  to  a  subset  5  of  S  is  ''5  e  S'' .  The 
partial  order  is  given,  by  set  inclusion,  A  is 
the  operation  of  intersection,  and  v  is  the 
operation  of  union. 

Definition  1  Lei  L\  and  L2  he  Lattices  and  lei  I  be  a  map  from  Li  to  L^- 
We  will  say  that  I  is  an  inference  1/ 1  is  ord^r  preserving. 


An  inference  therefore  preserves  the 
logical  implication  of  statements.  The  map 
X-'^  from  the  previous  section  is  an  example 
of  an  inference  from  the  lattice  of 
observations  P(A!f)  to  the  lattice  P(5) .  We 
also  constructed,  from  an  a  priori  distribution, 
a  random  variable  Z  taking  values  in  the  lat¬ 
tice  P{S).  We  may  extend  this  to  a  random  in¬ 
ference  Z  defined  on  the  Boolean  lattice  B  = 
{0,1}  by 


Z(l)  =  i 
Z(0)  =  4> 


Suppose  that 

Li4i,2 

is  a  random  inference,  we  define  for  e  £  h  and 
h  £  L2 

mi{h)  =  Proh{I{e)  =  h) 
the  distribution  of  I.  Suppose 
that  Li  ^  L  and  L2  ^  L  are  two  inference  maps . 
We  can  combine  h  and  I2  by  taking  either  the 
conjunction  or  the  disjunction  of  Ji 
and  I2.  We  first  form  the  product  lattice  Xi  x 
L2  and  define  the  maps 


ii  X  L2  L,. 
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and 


by 


L\  X  £<2 


/iV/a 


L, 


/i  A  £2(61, 62)  =  /i(fii)  A  £2(62) 
£iV£2(ei,e2)  =  £i(ei)  V  £2(62). 


There  is  also  a  distribution  version  of 
these  formulas.  If  we  assume  that  £1  and  h 
are  independent  then 

hi/\.h^—h 

To  normalize  the  equation  we  condition  off 
the  event  £1  A£2(«i,e2)  =  .  If  we  set 

k=  £ 

hiAh:,zz<l> 

then  we  have  the  normalized  combination 
formula . 

=  (-r^)  (1) 

^  A,A/ia=/» 

Refering  to  the  previous  section,  if  Li  =  P(M), 
L2  -  B,  h  =  and  h  =  Z  tl^n  (1)  is  Bayes 

formula.  For  h  =  P{M)r  =  P{M),  h  =  and 


J2  =  IT^  then  (1)  is  Dempster's  rule  of  combi^ 
nation.  Equation  (1)  can  of  course  be 
extended  to  multiple  sources  or  inferences . 
Since  it  is  the  distributional  version  of  a 
logical  combination,  all  logical  properties 
are  preserved  by  the  distribution.  In  this 
way  one  gets  the  associative  law. 


Suppose  that  Li  ^  L  and  Li  ^  L  are_  two  in¬ 
ference  maps  defined  on  the  same  lattice  Li . 
For  0<A<1  we  define  the  mixture 

£  =  l£i  +  (1  —  ■^)£2 


by 


hie) 

hie) 


if  C/<A 
if  U>X 


where  U  is  an  independent,  uniform  [0,1]  random 
variable.  Notice  that  the  distribution  of 
the  mixture  is  the  mixture  of  the 
distributions. 


By  taking  various  combinations  of 
mixtures,  conjunctions,  and  disjunctions,  we 
get  a  plethora  of  combination  formu¬ 
las,  For  instance,  given  Li  h- L  and  L2 L,  we 
can  define 

Q  =  XlAJ  +  il-X)lVJ 

The  interpetation  of  Q  is  that  it  lies 
somewhere  between  choosing  to  consider  both  £ 
and  J  are  correct  and  at  least  one  of  £  or  J 
is  correct . 


We  can  also  define 

=  \I  +  il-X)e 

where  $  from  Li  to  L  is  constantly  equal  to  0, 
mixes  £  with  a  trivial  inference. 

The  reason  for  considering  lattices  is 
to  allow  for  more  general  structures  that  do 
not  require  as  many  hypothesis  and  conse¬ 
quently  have  a  reduced  computational  load. 
For  example  consider  the  lattice  of  subsets 
of  a  set  5,  1,  consisting  of  the  empty  set, 
the  set  5,  and  the  singleton  sets  and  define 
the  following  inference  from  P(5)  to  L 


f  B  if  B  is  empty  or  a  singleton 
JiB)  -  I  5  otherwise 


There  is  an  obvious  reduction  in  the 
number  of  hypothesis  to  consider,  but  with  a 
corresponding  penalty.  Information  about  B 
is  lost  unless  5  is  a  singleton.  Although 
this  example  is  extreme  it  is  useful  when  we 
are  making  measurements  that  directly 
correspond  to  states  and  we  have  a  minimal 
amount  of  statistical  information.  The  is 
illustrated  in  the  next  example. 


EXAMPLE:  THE  FAULTY  SENSOR  II 

Let  S  -  and  let  the  measurment  space 

M  equal  S.  Suppose  that  we  have  a  sensor 
that  is  to  determine  the  state  of  the  system, 
that  is,  it  returns  values  We  make 

the  assumption  that  with  probability  p  the 
sensor  accuractly  assesses  the  state  and  is 
otherwise  uncorrelated  with  the  actual  state. 

Under  these  conditions  we  see  that  as  a 
lattice  the  measurement  space  as  well  as  the 
hypothesis  space  can  be  represented  by  L,  We 
define  now  the  random  inference  map  from  L  to 
itself  as  follows 

J(S)  =  5 

,,  ,,  f  {«}  if  the  sensor  is  accurate 
=  [  S  otherwise 

JW  = 

Of  course  we  do  not  know  whether  the  sensor 
is  working  or  not,  but  we  do  know  the 
probability, 

Prob  (/({«})  =  {«})  =  P 
Proh  iJi{s})  =  S)  =  1-p. 

Notice  that  J  =  /p  in  distribution  where  £ 

L -*■  L  is  the  identity  map.  We  can  use  our  up¬ 
date  formula  (1)  to  combine  two  independent 
measurements.  An  advantage  of  this  approach 
is  that  a  single  number  is  used  to  measure 
confidence  in  the  sensor. 


In  order  to  determine  what  value  we 
should  assign  to  p,  we  can  think  of  perform¬ 
ing  a  series  of  independent  experiments .  The 
results  of  the  experiments  will  be  a  series 
of  states  5i,52,....  Assume  that  the  system  is 
in  state  sk,  and  there  exists,  unbeknownst  to 
us,  conditional  probabili¬ 
ties  pi,P2,.--,PiV/  pj  =  Prob{5n  =  sj)  .  If  we  compute 
the  ratios  of  the  plausibilities  after  n  of 
experiments  we  find 


Pl(5fc) 

Plisi) 


(1 


where  Nj  is  the  number  of  times  that  the 
sensor  reported  state  sj .  if 


0  <  p  <  1  —  Q! 

where  a  =  supj^kiPj/Pk)/  then  PI  (sy ) /Pi  (5*)  0  for 

j^k.  if 

p=  1  -  s/a 

then  we  have  optimized  the  rate  of 
convergence  in  the  sense  of  expected  values. 
If  one  wishes  to  determine  how  many  samples 
must  be  taken  in  order  to  ensure  that 

Pr  Ob  ( ^  I  <  r  V  J  fc )  >l-€ 
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where  e  is  a  small  number,  then,  using  the 
above  value  of  p,  we  must  have 

where  ^  =  supj^jt  py  +  p*  -  ‘l./vjn 

To  compare  this  with  a  purely  Bayesian 
inference,  we  suppose  that  we  have  estimated 

Prob{Sj  =  sib(5ib)  =p 

with  the  rest  of  the  mass  being  distributed 
over  the  remaining  states  equally.  In  this 
case,  in  order  to  have  convergence  in  mean, 
we  must  have 

1  ^  ^  1 
AT  a(Ar-i)+i’ 

with  the  optimal  value 

1 

VJ^{jv-i)  +  i 


We  see  that  choosing  this  probability 
for  the  purely  Bayesian  inference  depends 
intimately  on  the  number  of  states  in 
question.  In  addition,  there  is  a  narrower 
range  of  acceptable  values 

that  may  be  used.  For  example  if  AT  =  100  and 
a  =  .5,  which  means  that  the  state  is  twice 
as  likely  as  any  other  state,  then  from  the 
Bayesian  viewpoint  we  must  have 

0.01  <  p  <  0.02, 

while  for  our  inference  we  only  need 
0.0  <  p  <  0.5 

to  have  convergence.  The  optimal  values  are 
0.014  for  the  Bayesian  inference  and  0.3  for  our 
first  inference. 

In  addition,  one  still  has  the  option  in 
either  case  to  combine  the  result 
with  an  a  ■priori  distribution,  as  represented  by 
an  inference  Z  from  the  boolean  lattice  to  L, 
It  is  also  possible  to  combine  with 

for  instance,  if  the  apriori  distribution  is 
a  statistic  based  on  only  a  percentage  of  the 
population . 

In  general,  if  A  c  P(5)  is  a  collection  of 
subsets  closed  under  intersection,  then  we 
can  turn  L  into  a  lattice  together 
with  an  inference  J\P{S)-^L.  This  will  allow 
the  tailoring  of  the  lattice  to  the  desired 
hypotheses  that  need  to  be  tested.  In  order 
to  compute  the  distribution  of  the  combined 
inference  from  P{M)  to  L  we  need  to  model  the 
information  that  is  contained  in  a 
measurement  with  respect  to  Z. 


SUMMARY 

In  the  first  section  it  was  shown  that 
by  considering  inferences  one  can  refer  to 
either  Bayesian  or  evidential  reasoning, that 
it  is  possible  to  combine  Bayesian  and  evi¬ 
dential  reasoning  in  the  sense  that  starting 
with  partial  Bayesian  information,  i.e. 
conditional  probabilities,  one  can  construct 
mass  functions  on  sets  as  in  Dempster-Shafer 
theory,  and  that  the  mass  distribution 
when  combined  with  an  a  priori  distribution,  was 
in  agreement  with  Bayes  formula.  It  was  also 
shown  that  the  desired  update  formulas  could 
be  interpreted  as  logical  combinations  of 


inferences  and  that  a  consistent  statistical 
measure  can  be  applied  to  interpret  the 
results . 

In  the  second  section  we  abstracted  the 
notion  of  an  inference  from  subsets  to 
lattices  and  developed  a  corresponding 
combination  formula.  By  considering  lattices 
we  are  freed  from  having  to  construct  a  mass 
distribution  on  the  full  power  set  and  an  ex¬ 
ample  was  given  that  had  a  minimal  number  of 
hypotheses.  Finally  we  compared  this  example 
to  the  corresponding  Bayesian  situation. 

One  question  that  comes  up  is  that  of 
the  normalization.  This  has  a  ready 
interpretation  since  the  normalization 
factor  is  PI  (^)  .  Logically  this  means  that 
we  are  conditioning  off  of  'False' .  We  do 
this  if  we  believe  that  our  assumptions  are 
correct  probabilistically.  If  PI  (0)  is  below 
some  threshold,  then  we  are  conditioning  on 
an  event  of  low  probability;  this  is  a  signal 
that  we  are  combining  erroneous  data.  Thus, 
the  normalization  factor  serves  as  an  error 
warning.  This  can  also  serve  as  a  measure  of 
agreement  when  combining  data  from  many 
sources 

It  is  also  possible  to  measure  the 
amount  of  statistical  information  and  at 
various  levels.  Prob(/  =  6>)  measures,  in  a 
sense,  the  total  ignorence  of  the  inference 
mechanism.  Note,  that,  using  this 
definition,  a  Bayesian  system  has  complete 
statistical  information  when  one  uses  the  a 
priori  distribution. 

It  is  hoped  that  the  observations  in 
this  paper  will  help  to  quell  the  levels  of 
•controversy  of  whether  one  should  perform 
Bayesian  verses  evidential  reasoning,  in  that 
they  can  both  be  considered  as  inference 
mechanisms.  The  important  question  is  how  to 
statistically  model  the  information  in  a  data 
source.  A  few  general  examples  were  given, 

but  they  are  foundational  in  nature.  SpeT 
cific  examples  require  more  in  depth  study, 
such  as  the  generalized  tracking  problem 
where  the  system  evolves  with  time  as 
measurements  are  taken.  It  is  possible  to 
treat  the  evolution  of  states  also  as  an 
inference,  an  inference  that  has  less 
statistical  information  as  time  increases, 
i.e. 

Prob  {It  =  e) 

increases  with  time. 

As  a  final  observation,  we  note  that  the 
space  of  fuzzy  sets  is  also  a  lattice.  Of 
course,  now  it  is  no  longer  finite  so  that 
the  problem  of  working  over  a  continuous  lat¬ 
tice  is  presented.  Although  the  results  are 
not  conclusive,  preliminary  work  indicates 
that  in  order  to  work  with  infinite  lattices, 
additional  structure  is  desired;  for  example, 
we  may  require  our  inference  to  satisfy  a 
continuity  property.  This  is  an  as  yet 
unexplored  area  that  awaits  investigation. 
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Abstract 


Definitions  are  reviewed  of  report-to-report  and  track-to-track  correlation  with 
respect  to  target  tracks  in  surveillance  systems-  The  need  for  track-to-track  correlation 
in  merging  and  linking  tracks  is  indicated.  An  algorithm  for  automatic  decision  making  in 
merging  and  linking  tracks  is  derived  from  fundamentals  of  target  location  information  in 
reports  received  by  surveillance  systems.  The  algorithm  is  applied  successfully  to  data 
that  are  representative  of  the  systems.  .  The  illustrative  examples  demonstrate  algorithm 
usefulness  not  only  for  track-to-t'rack  correlation,  but  also  for  report-to-track 
correlation. 
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1.  INTRODUCTION 

The  function  of  some  surveillance  sys¬ 
tems  is  the  representation  of  target  trades 
derived  from  reports  received  from  various 
sources.  The  reports  contain  two  kinds  of 
information  about  the  targets:  (a)  location 
of  a  target  at  a  specific  time;  and  (b)  non- 
locational  information,  such  as  the  vessel 
type,  identifying  physical  attributes,  or 
electronic  emission  characteristics. 

Surveillance  systems  algorithms  use  both 
kinds  of  information  to  sort  reports  by  tar¬ 
get,  and  to  string  the  locations  together  to 
represent  the  track  of  the  target.  The  pro¬ 
cess  of  assigning  the  information  of  a  report 
to  a  track  is  known  as  "report-to-track  cor¬ 
relation".  Surveys  of  such  correlation  algo¬ 
rithms  are  given  in  reports  published  by  the 
Naval  Research  Laboratory.  [1,2] 

Report-to-track  correlation  can  assign  a 
report  to  one  track  only,  but  cannot  guaran¬ 
tee  that  a  platform  is  associated  with  just 
one  track  given  a  sequence  of  reports.  In 
fact,  the  known  report-to-track  correlation 
algorithms  will  frequently  yield  a  multipli¬ 
city  of  tracks  per  platform.  That  condition 
can  occur  when  a  platform  is  being  tracked  by 
more  than  one  sensor.  It  can  occur  also  when 
an  emitter-tracked  platform  activates  more 
than  one  emitter,  or  operates  an  emitter  in 
different  modes. 

An  algorithm  due  to  Reid  [3]  and  an 
extension  of  that  algorithm  to  process  data 
from  multiple  sensors  [4]  have  been  success¬ 
ful  in  decreasing  the  multiple  track  repre¬ 
sentation  of  single  tracks  by  a  technique  for 
fusing  data  on  the  same  target  received  from 
multiple  sensors.  However,  the  technique 
cannot  remove  all  ambiguities,  particularly 
when  there  are  large  time  gaps  in  the 
sequence  of  reports,  and  when  reports  arrive 


out  of  sequence.  Preliminary  tests  show  that 
the  technique  can  segment  a  single  track, 
representing  the  segments  as  different 
tracks. 

"Track-to-track  correlation"  is  required 
to  reduce  the  multiple  track  representation 
of  a  platform  to  single  track  representation. 

This  paper  presents  a  method  for  linking 
the  segments  of  the  same  track,  and  for  merg¬ 
ing  multiple  representations  of  a  single 
track.  The  paper  derives  a  track-to-track 
correlation  algorithm  from  (a)  the  ergodicity 
of  the  process  formed  by  measurements  of  a 
moving  target  taken  by  different  sensors  and* 
(b)  the  change  induced  is  a  best  estimate  of 
the  track  when  one  set  of  measurements  is 
merged  with  another  set  that  may  or  may  not 
be  attributable  to  the  same  target. 

The  successful  application  of  the  algo¬ 
rithm  to  track  merging  and  linking  shows  the 
potential  of  the  method  as  a  much  needed 
supplement  to  current  tracking  techniques. 
Weighted,  least  squares  splines,  referenced 
in  the  paper,  are  used  as  readily  implemen- 
table  low-pass  filters. 

A  formal  statement  of  the  problem,  and  a 
derivation  of  the  algorithm  as  a  solution 
fol  lows . 

2.  PROBLEM 

Statement  of  the  Problem 

From  non-locational  information  about 
two  targets,  each  associated  with  a  distinct 
track  formed  by  measurements  of  different 
sensors,  and  from  the  times  of  the  measure¬ 
ments,  it  is  known  that  the  targets  may  not 
be  different.  Assume  that  the  times  are 
error-free,  and  that  for  the  position  data, 
uncorrupted  by  systematic  error,  there  are 
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estimates  of  the  measurement  error.  Find  a 
measure  of  correlation  between  the  targets. 

Discussion 

Track  of  Target.  A  track  is  a  collec¬ 
tion  of  geographic  points  that  represent 
positions  of  a  moving  platform,  known  as  a 
target.  Sensor  measurements  of  positions 
form  a  probability  distribution,  the  first 
moment  of  which  is  a  function  of  time.  If 
sensor  measurements  are  free  of  systematic 
error,  the  first  moment  is  a  best  estimate  of 
the  track. 

When  measurements  of  a  target  made  by 
two  or  more  sensors  are  combined,  the 
resultant  combination  of  distributions  forms 
a  stochastic  process.  If  every  sensor  con¬ 
tributing  to  the  process  is  free  of  syste¬ 
matic  error,  and  every  sensor  is  measuring^ 
the  same  target,  the  process  mus:^  be  ergodic. 
That  is,  the  first  moment  of  a  distribution 
must  represent  "the  same"  track  as  the  first 
moment  of  the  ensemble.  And  if  the  distribu¬ 
tion  corresponding  to  one  target  is  combined 
with  that  corresponding  to  a  different^ 
target.  The  first  moment  of  one  distribution 
and  that  of  the  ensemble  represent  tracks 
that  are  "not  the  same". 


Practical  definitions  of  "the  same"  and 
of  "not  the  same"  require  analysis  of  a  rela¬ 
tively  large  body  of  empirical  data.  Pending 
such  an  analysis  it  is  possible  to  set  tenta¬ 
tive  definitions  by  analogy  with  performance 
specifications  for  communication  filters,  or 
by  analogy  with  an  acceptable  finite  series 
approximation  to  a  function.  Noting  that  in 
this  case  the  first  moments  are  deterministic 
functions,  the  first  moment  of  a  distribution 
and  the  first  moment  of  an  ensemble  may  be 
said  to  be  "the  same"  if  the  functions  are 
within  3  dB  of  each  other,  and  "not  the  same" 
if  they  are  at  least  6  dB  apart. 

Model  of  Track.  The  actual  track  is  of 
a  form  that  can  be  modeled  by  a  mathematical 
expression  suitable  for  best  estimate  tech¬ 
niques.  For  example,  the  track  can  be 
modeled  by  a  velocity  vector  or  by  means  of 
polynomials  fitted  to  intervals  in  a  least 
squares  sense.  A  velocity  vector  model  has 
been  customary  in  tracking  with  the  aid  of 
Kalman  type  filtering.  Justification  for  a 
spline  model  follows. 

Weighted  Least  Squares  Spline.  Consider 
that  the  targets  being  tracked  can  change 
direction  no  faster  than  aircraft.  The 
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Figure  2 

Small  Displacement  from  Best  Estimate 
Induced  by  Reports  from  Same  Platform 


tracks  reconstructed  by  the  surveillance 
system  are  composed  of  sampled  target  loca¬ 
tions.  Say  that  the  spacing  of  samples  is 
such  that  they  include  at  least  one  point  of 
a  maneuver.  Then  the  tracks  are  piecewise 
representable  by  polynomials  of  degree  no 
higher  than  three.  That  is,  the  report  posi¬ 
tions  can  be  used  as  the  knots  of  a  cubic 
spline.  Where  an  estimate  of  the  measurement 
error  accompanies  the  reported  location,  a 
weighted  least  squares  cubic  spline  [5]  can 
be  used  to  represent  the  target  track  in  seg¬ 
ments,  with  the  endpoints  of  a  segment  coin¬ 
ciding  with  a  pair  of  adjacent  knots.  There 
is  a  weighted,  least  squares  spline  represen¬ 
tation  [6]  such  that  a  segment  is  represent¬ 
able  by  a  closest  fit  polynomial,  of  first, 
second  or  third  degree,  whichever  is  most 
applicable.  This  representation  is  equiva¬ 
lent  to  a  smoothing  or  low-pass  filter.  [7] 

Best  Estimates,  Target  Location,  Loca¬ 
tion  Error.  From  either  the  Kalman  type 
filtering  or  the  weighted,  least  squares 
spline  fit  to  the  position  data,  there  are 
obtainable: 

(a)  A  best  estimate  of  the  true  location 
at  the  time  of  the  measured  location,  and 


(b)  A  best  estimate  of  the  error  in  the 
measured  location.  Both  are  useful  in  com¬ 
puting  the  correlation  of  measurements. 

Displacement  of  Best  Estimate  Track , 
Consider  two  sets  of  -  reports,  each  possibly 
associated  with  the  same  target.  Call  one 
set  the  Tester,  the  other  set,  the  Candidate. 
The  Tester  set  is  one  distribution  of  meas¬ 
urements.  Find  the  first  moment,  making  a 
best  estimate  of  the  Tester  track,  and  note 
the  estimated  errors  (residuals).  Merge  the 
Tester  and  the  Candidate  sets  of  reports, 
forming  a  process.  Call  the  result  the 
Mergence.  Find  the  first  moment  of  the 
Mergence,  making  a  best  estimate  of  the 
Mergence  track. 

The  best  estimate  Mergence  track  con¬ 
tains  a  subset  of  positions  that  correspond 
to  the  positions  of  the  original  best  esti¬ 
mate  Tester  track.  Call  this  subset  the 
Displaced  Tester. 

The  correlation  of  Candidate  to  Tester 
can  be  determined  by  comparing  the  energy  of 
the  function  representing  the  Tester  with 
that  of  the  function  representing  the  Dis¬ 
placed  Tester.  Alternatively,  the  correla¬ 
tion  can  be  found  from  the  displacement  of 
the  Tester. 
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For  the  alternative  technique,  find  the 
distance  between  each  point  of  the  Tester  and 
the  corresponding  point  of  the  Displaced 
Tester.  This  displacement  in  the  original 
best  estimate  Tester  track  is  induced  by  the 
Candidate  track.  If  the  displacement  is 
negligible,  the  process  can  be  considered 
ergodic;  or  Tester  and  Candidate  tracks 
judged  to  be  due  to  the  same  target.  If  the 
displacement  is  large.  Tester  and  Candidate 
tracks  most  likely  were  due  to  different 
targets . 

Displacement  Illustrations.  Figures  1 
and  2  illustrate  the  displacement  principle. 

In  the  illustrations  target  positions  are  in 
geographic  coordinates,  with  time  of  loca¬ 
tions  and  values  of  longitudes  error-free. 

In  Figure  1,  Target  A  is  different  from 
Target  B.  In  Figure  2,  Targets  C  and  D  are 
the  same.  In  each  illustration,  the  two 
tracks  are  due  to  independent  sources  of 
observations . 

Figure  lb  shows  the  measured  locations 
and  the  corresponding  best  estimate  locations 
of  the  track  for  Target  A,  the  Tester. 

Figure  Ic  compares  the  Tester  with  the  best 
estimate  obtained  when  observations  from 
Targets  A  and  B  are  merged.  The  displacement 
is  noticeably  large,  indicating  that  Target  A 
is  different  from  Target  B. 

A  similar  comparison  is.  shown  in  Figures 
2b  and  2c.  Here  the  induced  displacement  of 
the  Tester  track  positions  is  minor,  indi¬ 
cating  that  Target  C  is  the  same  as  Target  D. 

Displacement,  Measurement  Errp_r_,_and 
Correlation.  The  displacement  is  error  in 
the  Tester  track  disclosed  by  or  resulting 
from  the  merging  of  Candidate  and  Tester. 

The  total  error  in  the  Tester  track  after  the 
merging  is  the  sum  of  the  measurement  error 
and  the  displacement.  Comparison  of  Tester 
track  measurement  error  with  the  total  error 
in  the  track  after  merging  can  be  made  equiv¬ 
alent  to  a  comparison  of  the  original  Tester 
track  with  the  displaced  track  for  the  pur¬ 
pose  of  computing  correlation.^  When  the 
track  is  on  a  curved  surface,  it  is 
convenient  to  compute  the  measure  of  correla¬ 
tion  in  terms  of  the  values  of  the  errors, 
which  are  generally  much  smaller  than  the 
values  of  the  track  coordinates. 

3.  MEASURE  OF  Cp_RRELATION 
Tester  Error 

At  some  time  ,  let  T^,  the 

measured  location  of  a  moving  target  be 
accompanied  by  an  estimate  of  the  uncer¬ 
tainty  in  the  measurement. 


where  is  the  true  location,  e ^  is  the 

error  in  the  measurement  of  time  and 

all  three  terms  are  vectors.  If  there  is  no 
systematic  error,  the  mean  of  the  measurement 
error  is  0.  Then  the  mean  of tT^  equals 

P.  . 

1 

Collect  a  set  of  n  measurements,  )t^, 
i=l,2,...nj.  Find  P^,  the  best  estimate  of 

P.  )i=l , 2, . . . , n) .  The  best  estimate  of 
tie  error  is  the  residual 


Change  Induced,  by_ Candidate 

At  some  time  ^ ,  let  the  measured 
location  of  a  moving  target  made  by  another 

sensor  by  C.,  accompanied  by  an  estimate  of 
the  uncertainty  in  the  measurement.  Collect 

a  set  of  m  measurements  |C ^ ,  j=l,2,...,mj 
and  add  to  the  jTj  set.  Find  the  best 

estimate  of  the  track  represented  by  |T^  + 

C .!  ,  and  select  the  points  jQ^, 

1  =  1 , 2 ,  .  . .  ,.nl  that  correspond  to  the  points 
*  ^i '  ^  best  estimate  of 

P..  The  new  best  estimate  is  displaced 
from  the  old  by 


Measure  in  Terms  of  Best _ E s t J. mate  Tracks 

The  energy  of  the  first  best  estimate  of 
jPil  is 


that  of  the  new  best  estimate. 


Formally,  the  measure  of  correlation  can  be 
2 

written  as  M  , 

=  Ej/E2  . 

The  estimates  are  within  3  dB  of  each  other 
when  =  0.5  ,  and  are  at  least  6  dB  apart 
when  =0.25  . 

Where  target  location  coordinates  are  in 
earth  latitude  and  longitude,  calculation  of 
the  terms  composing  and  E2  depends  on 

the  earth  model  used.  The  simplest  earth 
model  to  use  is  a  sphere.  But  where  the 
track  extends  over  hundreds  of  miles,  the 
spherical  earth  model  can  introduce  signifi¬ 
cant  error  in  the  calculations.  More 
accurate  earth  models  complicate  the  calcula¬ 
tions  considerably. 

Relative  to  the  latitude/longitude  com¬ 
ponents  of  target  location,  residuals  and 
displacements  are  small  enough  for  their 
energies  to  be  calculated  with  the  use  of  a 
spherical  earth  model.  Therefore,  it  is 
convenient  to  write  the  measure  of  correla¬ 
tion  in  terms  of  residuals  and  displacements. 

Let  R  be  the  root -mean-square  (rms) 
value  of  the  set  of  residuals  j^, 

i  =  l,2,.  ..,n)j,  and  D,  the  rms  value  of  the  set 
of  displacements  i  6 ^ ,  i  =  l , 2 , .  . . , nj  .  R" 
and  D^  are  defined  by 


R^  =  (1/n)  and 

i  =  l  ^ 


D^  =  (1/n)  E.  (  6 J  ' 
i  =  l 


R  +  D  is  the  estimated  total  error  in  the 
Tester  after  the  merge. 
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Figure  3 

Calculation  of  Measure  of  Correlation  for 
2-Diinensional  Cartesian  Coordinates 


The  measurement  error  relative  to  the 
total  error  is  M, 

M  =  R/{R  +  D) 

-  1/[1  +  (D/R)] 

=  [1  +  (D'/R/)]‘^ 

where  D'^  =  nD^,  and  R'^  =  nR^.  That 
is,  the  root-sum-square  (rss)  values  of 
residuals  and  displacements  can  be  substi¬ 
tuted  for  the  rms  values  in  the  calculation 
of  M. 

Note  that  M  (which  compares  the  measure¬ 
ment  error  of  the  Tester  track  function,  with 
the  total  error  of  that  track  function  after 
the  merge  of  Tester  with  Candidate  track) 
varies  directly  with  the  square  root  of  the 
ratio  of  the  energy  of  the  two  functions. 
Pending  empirically  determined  thresholds,  it 
can  be  expected  that  the  functions  are  the 
same,  or  that  Tester  and  Candidate  tracks 
belong  to  the  same  target  when  M  =  3  dB;  and 
that  they  belong  to  different  targets  when 
M  =  6  dB.  Equivalently, 

if  M  =  0.7,  accept  correlation  of  the 
targets , 

if  M  =  0.5,  reject  correlation  of  the 
targets. 

If  0-5  <  M  <  0.7,  correlation  of  the  targets 
is  uncertain. 


4.  CALCULATION  OF  MEASURE 
Cartesian  Coordinates 

Calculation  of  the  measure  using  rss 
error  is  diagrammed  in  Figure  3  for  locations 
given  in  two-dimensional  Cartesian  coordi¬ 
nates.  The  extension  to  three  dimensions  is 
obvious . 

The  best  estimate  calculation  is 
effected  in  stages,  one  for  each  dimension  of 
the  location.  Each  dimension  is  accompanied 
with  the  appropriate  component  of  uncer¬ 
tainty.  For  example,  the  x-component  Tx. 
is  accompanied  with  o-Tx^ .  ^ 

Geographic  Coordinates 

The  calculation  in  stages  is  applicable 
to  locations  of  targets  on  the  surface  of  the 
earth.  Each  location  is  given  as  a  latitude- 
longitude  pair  together  with  an  uncertainty 
ellipse  defined  by  semi-major  and  semi-minor 
axes  in  nautical  miles,  plus  orientation  of 
the  major  axis  relative  to  North.  Allocation 
of  uncertainty  to  latitude  and  to  longitude 
is  obtainable  from  the  ellipse  parameters  by 
techniques  for  coordinate  system  change.  The 
residual  (or  displacement)  error  can  be  cal¬ 
culated  as  the  hypotenuse  of  the  spherical 
right  triangle  with  legs  equal  to  the 
residuals  (or  displacements)  in  latitude  and 
longitude . 
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LATITUDE 


Plot  of  Three  Closely  Spaced  Tracks 


TABLE  I 

Locations  for  Simulated  Target  SI 3 


S 

E 

G 

M 

E 

N 

T 

S 


Time  Location  Uncertainty, 

k  /u  T.at  Lon  a  b  0 


D/H/M 

Lat 

Lon 

a 

b 

d 

0/14/55 

4030 

-1002 

1 

0 

87 

0/19/29 

4139 

-1026 

12 

6 

74 

1  1/04/06 

4343 

-1007 

1 

0 

12 

1/06/44 

4414 

-0937 

3 

0 

52 

1  1/16/07 

4606 

-0739 

10 

5 

20 

1/19/52 

4701 

-0701 

12 

8 

28 

^  1/20/39 

4709 

-0650 

1 

0 

4 

2/00/21 

4754 

-0603 

2 

2 

37 

2/05/55 

4802 

-0433 

94 

38 

58 

2/11/45 

4945 

-0255 

4 

1 

27 

2/15/24 

4943 

-0206 

7 

1 

5 

2/19/53 

5003 

-0109 

2 

1 

32 

3/03/20 

5024 

00029 

24 

7 

0 

3/05/11 

5021 

00054 

1 

0 

30 

1  3/06/09 

5028 

00057 

8 

1 

1 

[3/09/04 

5040 

00115 

2 

1 

13 

3/17/22 

51 18 

00212 

3 

2 

39 

(3/18/23 

5127 

00215 

13 

11 

53 

3/20/58 

5150 

00229 

1 

0 

98 

[4/05/27 

5309 

00325 

1 

0 

65 

14/18/30 

5509 

00446 

1 

0 

2 

4/22/09 

5543 

00509 

3 

2 

59 

Time;  D/H/M  =  Days /Hours/Minutes  lapsed  from  zero  time 
Location:  Latitude,  Longitude  in  Degrees  and  Minutes 
Uncertainty  Ellipse;  a  =  semi-major  axis  in  nmi. 

b  =  semi-minor  axis  in  nmi. 

B  =  orientation  angle  in  degrees 
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MERGING/UNKING 


LATITUDE 


LATITUDE  VS.  LONGITUDE  S13  TRACK 


55.7166672 

53.8145828 

51.9125023 

50.0104179 

48.1083336= 

46.2062492 

44.3041687 

42.4020844 

40.5000000  = 


LONGITUDE 


Figure  5 

Segmentation  of  Track  S13 


TABLE  II 

Locations  for  Simulated  Target  113 


Time  Location  Uncertainty 


D/H/M 

Lat 

Lon 

a 

b 

e 

0/20/21 

4155 

-1032 

18 

9 

167 

1/04/39 

4353 

-0959 

13 

10 

88 

1/05/50 

4348 

-0948 

21 

6 

1 

1/16/39 

4620 

-0745 

14 

2 

69 

1/18/53 

4644 

-0705 

16 

9 

11 

1/20/20 

4702 

-0707 

8 

5 

7 

1/23/51 

4807 

-0619 

25 

6 

2 

2/05/15 

4853 

-0458 

8 

7 

111 

2/11/54 

4944 

-0258 

5 

3 

79 

2/15/30 

4954 

-0204 

23 

13 

102 

2/20/20 

5008 

-0105 

8 

4 

1 

3/03/15 

5024 

00039 

13 

13 

0 

3/05/18 

5024 

00024 

17 

14 

83 

3/06/51 

5026 

00103 

4 

0 

2 

3/08/30 

5037 

00113 

10 

1 

7 

3/16/36 

51 13 

00200 

1 

0 

156 

3/18/18 

5123 

00212 

7 

5 

1 

3/21/51 

5152 

00239 

18 

5 

68 

4/05/17 

5309 

00313 

23 

17 

11 

4/17/42 

5459 

00443 

13 

3 

9 

Time:  D/H/M  =  Days /Hours/Minutes  lapsed  from  zero  time 
Location:  Latitude,  Longitude  in  Degrees  and  Minutes 
Uncertainty  Ellipse:  a  =  semi-major  axis  in  nmi . 

b  =  semi-minor  axis  in  nmi. 

6  =  orientation  angle  in  degrees 
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MERGIN/LINKING 


LATITUDE 


LATITUDE  VS.  LONGITUDE  SI  TRACK 


Segmentation  of  Track  SI 


TABLE  III 

Locations  for  Simulated  Target  P13 


Time  Location  Uncertainty 


D/H/M 

Lat 

Lon 

a 

b 

6 

0/19/37 

4214 

-1010 

8 

8 

15 

1/04/24 

4420 

-1001 

7 

4 

2 

1/06/47 

4449 

-0941 

12 

8 

54 

1/17/28 

4656 

-0736 

16 

10 

3 

1/18/41 

4716 

-0717 

6 

1 

8 

1/20/56 

4745 

-0634 

12 

10 

39 

2/00/05 

4818 

-0624 

11 

8 

17 

2/07/18 

4949 

-0359 

7 

4 

63 

2/12/07 

5017 

-0300 

16 

3 

23 

2/14/33 

5020 

-0231 

16 

9 

1 

2/21/09 

5034 

-0055 

16 

15 

142 

3/03/48 

5053 

00041 

3 

2 

13 

3/03/55 

5052 

00045 

5 

5 

12 

3/06/24 

5058 

00046 

13 

9 

32 

3/11/20 

51 11 

00133 

13 

8 

34 

3/17/10 

5144 

00201 

12 

5 

121 

3/18/10 

5200 

002  29 

6 

4 

20 

3/21/38 

5228 

00232 

9 

2 

5 

4/06/21 

5353 

00330 

6 

1 

35 

4/18/57 

5548 

00448 

2 

1 

117 

Time:  D/H/M  =  Days/Hours/Minutes  lapsed  from  zero  time 
Location:  Latitude,  Longitude  in  Degrees  and  Minutes 
Uncertainty  Ellipse:  a  =  semi-major  axis  in  nmi . 

b  =  semi-minor  axis  in  nmi. 
d  ~  orientation  angle  in  degrees 
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MERGING/LINKING 


TABLE  VI 

Measure  of  Correlation  for  Segment  Linking 


Tester 

Candidate 

Computed  Measure 

Interpretation 

S13A 

S13B 

0.689 

Link  uncertain 

S13A 

S13C 

0.843 

Accept  link 

S13A 

S13D 

0.835 

Accept  link 

S13A 

S13E 

0.841 

Accept  link 

S13ACDE 

S13B 

0.706 

Accept  link 

SIA 

SIB 

0.890 

Accept  link 

SIA 

SIC 

0.902 

Accept  link 

SIA 

SID 

0.756 

Accept  link 

S13ABCDE 

SID 

0.484. 

Reject  link 

TABLE  IV 

Measure  of  Correlation  for  Track 

Merging 

Tester 

Candidate 

Computed  Measure 

Interpretation 

S13 

P13 

0.495 

Reject  Merge 

S13 

113 

0.753 

Accept  Merge 

113 

P13 

0.500 

Reject  Merge 

TABLE  V 

Locations  for  Simulated  Target  SI 


Time 

Location 

Uncertainty 

D/H/M 

Lat 

Lon 

a 

b 

B 

0/00/00 

6852 

03721 

8 

3 

1 

0/12/11 

7018 

03606 

11 

5 

37 

0/16/47 

7030 

03503 

15 

4 

20 

0/17/45 

7035 

03457 

2 

0 

44 

1/02/55 

7109 

03159 

5 

2 

39 

1/04/06 

71 13 

03123 

1 

0 

42 

1/12/22 

7135 

02745 

1 

0 

24 

2/07/03 

7100 

01919 

4 

2 

44 

2/09/53 

7056 

01802 

2 

1 

50 

2/10/41 

7052 

01739 

3 

1 

18 

2/11/41 

7050 

01715 

1 

0 

16 

2/15/23 

7013 

01532 

1 

1 

57 

2/19/13 

6930 

01351 

6 

2 

24 

2/23/43 

6837 

01157 

21 

8 

46 

3/00/45 

68  24 

01125 

1 

0 

21 

3/01/27 

6816 

01107 

5 

4 

49 

3/03/20 

6754 

01018 

12 

5 

39 

3/13/38 

6546 

00654 

2 

1 

40 

3/17/32 

6454 

00543 

4 

4 

46 

3/18/16 

64  46 

00533 

4 

2 

68 

3/20/04 

6409 

00528 

23 

22 

42 

3/21/14 

6407 

00445 

11 

6 

34 

3/22/55 

6343 

00415 

1 

0 

48 

4/00/58 

6316 

00339 

5 

2 

0 

4/02/33 

6255 

00312 

2 

2 

60 

,  4/03/43 

6238 

00255 

5 

4 

18 

4/04/31 

6228 

00247 

4 

2 

48 

4/07/23 

6145 

00239 

7 

5 

22 

4/09/15 

61 15 

00244 

20 

7 

38 

4/12/59 

6018 

00251 

2 

1 

55 

4/19/21 

5845 

00255 

6 

3 

70 

4/21/21 

5815 

00244 

10 

9 

63 

4/2  2/10 

5805 

00252 

3 

1 

74 

Time:  D/H/M  =  Days/Hour s/Minutes  lapsed  from  zero  time 
Location:  Latitude,  Longitude  in  Degrees  and  Minutes 
Uncertainty  Ellipse:  a  =  semi-major  axis  in  nmi. 

b  =  semi-minor  axis  in  nmi. 

B  =  orientation  angle  in  degrees 
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5.  EXAMPLES 

The  use  of  the  Measure  of  Correlation  is 
illustrated  with  two  kinds  of  cases; 

a.  Merge  Test.  A  test  for  deciding 
whether  or  not  two  or  more  tracks  occurring 
in  approximately  the  same  time  interval  are 
associated  with  the  same  target. 

b.  Link  Test.  A  test  for  deciding 
whether  or  not  time  separated  tracks  (track 
segments)  are  associated  with  the  same 
target . 

In  both  tests  one  of  the  tracks  (or  seg¬ 
ments)  is  taken  to  be  the  Tester,  each  re¬ 
maining  track  (or  track  segment),  a  Candi¬ 
date.  Best  estimate  values  were  obtained  by 
the  weighted,  least  squares  cubic  spline 
developed  by  The  International  Mathematics 
and  Statistics  Library.  [6] 

For  each  position  an  estimated  one-sigma 
error  in  latitude  and  one-sigma  error  in 
longitude,  computed  from  the  uncertainty 
ellipse  parameters,  was  used  as  a  measured 
position  weight  in  the  best  estimate  lati¬ 
tude  and  best  estimate  longitude  stages, 
respectively.  For  the  smoothing  value 
required  by  the  program,  half  the  quantity  of 
input  points  was  used  per  stage. 

Merge  Test 

For  the  merge  test  three  closely  spaced 
tracks  were  simulated.  (See  Figure  4.)  The 
three  tracks  came  from  "error-free"  positions 
of  two  distinct  targets,  observed  over  the 
same  time  interval,  and  traveling  parallel 
courses,  one  36  nautical  miles  North  of  the 
other.  Track  P13  was  simulated  by  adding  ran¬ 
dom  error  and  an  uncertainty  ellipse  to 
sampled  positions  of  the  northern  track. 

Tracks  S13  and  113  were  similarly  formed  from 
the  other  track.  To  simulate  independent 
measurement  systems,  sampled  positions  of  one 
track  were  chosen  to  be  distinct  in  time  from 
those  of  each  of  the  other  tracks.  Simulated 
measurements  are  listed  in  Tables  I,  It,  and 
III  for  S13,  113,  and  P13,  resp.  Table  IV 
shows  the  measure  of  correlation  obtained 
with  each  of  the  other  tracks  when  track  Si 3 
is  taken  as  the  Tester.  The  computed  meas¬ 
ures  indicate  that  S13  and  113  are  tracks  of 
the  same  target,  and  that  P13  is  a  track  of  a 
target  different  from  S13  and  113. 

Link  Test 

For  the  link  test  two  tracks  were  each 
divided  into  disjoint  segments.  Table  I  and 
Figure  5  display  the  segmentation  of  simu¬ 
lated  Track  S13.  Table  V  and  Figure  6  display 
the  segmentation  of  simulated  Track  SI, 

Segment  A  of  Track  S13  was  taken  as  the 
Tester,  segments  B,  C,  D,  and  E,  as  Candi¬ 
dates.  The  link  test  results,  given  in  Table 
VI,  indicate  segments  A,  C,  D,  and  E  belong 
to  the  same  track.  There  is  uncertainty 
about  Candidate  B  being  linked  with  Tester  A. 
But  when  segments  A,  C,  D,  and  E  are  linked 
to  form  the  Tester,  with  segment  B  taken  as 
the  Candidate,  Candidate  B  is  accepted  for 
linking. 

Segment  A  of  Track  SI  was  taken  as  the 
Tester,  segments  B,  C,  and  D,  as  the  Candi¬ 
dates.  The  link  test  results,  given  in  Table 
VI,  indicate  that  all  segments  belong  to  the 
same  track. 


Finally,  segment  D  of  Track  SI  was  taken 
as  a  candidate  for  the  track  composed  of  the 
linked  segments  A,  B,  C,  D,  and  E  of  Track 
S13.  As  shown  in  Table  VI,  the  link  test 
rejected  the  candidate. 


6.  SUMMARY 

An  algorithm  was  presented  for  automat¬ 
ing  decisions  on  merging  and  linking  of  tar¬ 
get  tracks  as  represented  in  surveillance 
systems.  The  algorithm  supplements  estab¬ 
lished  tracking  techniques,  providing  the 
means  for  correcting  two  errors  common  to  the 
techniques . 

a.  Error  in  merging  tracks  occurring  in 
the  same  time  interval. 

b.  Error  in  linking  track  segments. 

Removal  of  these  errors  facilitates  the  cor¬ 
relation  of  one  track  with  another.  Applica¬ 
tion  of  the  algorithm  was  illustrated  with 
target  location  information  representative  of 
that  found  in  established  tracking  systems. 
The  examples  showed  that  the  algorithm  gives 
correct  decisions  on  the  correlation  of  track 
to  track,  and  that  the  algorithm  is  usable  in 
the  correlation  of  report  to  track. 


7.  AREAS  FOR  FURTHER  STUDY 

Accept an ce/Rej ect ion .  As  noted  above, 
the  acceptance/rejection  thresholds  were  set 
on  the  basis  of  theoretical  considerations, 
and  are  to  be  taken  as  subject  to  modifica¬ 
tion  by  results  of  a  field  study. 

Tester  Size.  The  algorithm  is  applied 
pairwise  to  sets  of  locations  representing 
tracks.  Because  the  method  supplements 
established  tracking  techniques,  at  least  one 
set  of  locations  attributed  to  a  target  will 
contain  two  or  more  elements.  A  set  that 
contains  a  single  element  cannot  be  chosen  as 
the  Tester. 

It  was  found  that  the  ambiguous  "uncer¬ 
tain"  decision  could  frequently  be  avoided 
when  the  larger  set  was  made  the  Tester.  In 
the  case  of  testing  for  track  merge,  no  limi¬ 
tation  on  the  Tester  size  relative  to  that  of 
the  Candidate  was  observed,  because  in  the 
simulations  the  sizes  of  the  sets  under  com¬ 
parison  were  of  the  same  order  of  magnitude. 
But  in  the  case  of  testing  for  the  linkage  of 
a  single  location  with  a  ‘set  of  locations 
covering  a  long  time  period  (many  days),  it 
was  found  advisable  to  restrict  the  size  of 
the  Tester  set  so  as  to  cover  a  shortened 
period  of  time  when  the  single  location  was 
geographically  close  to  the  Tester  track,  A 
similar  restriction  may  become  advisable  in 
testing  two  closely  spaced  sets  for  a  merge. 
That  restriction  depends  on  the  resolution  of 
the  method. 

Resolution.  After  the  establishment  of 
emphirically  determined  acceptance/rejection 
threshold  values,  it  would  be  useful  to 
determine  empirically  how  far  apart  a  pair  of 
parallel  tracks  must  be  for  the  method  to 
yield  a  correct  answer  in  the  merge  test. 

Detection  of  Incorrect  Link/Merge ,  Test¬ 
ing  of  the  algorithm  covered  many  more  cases 
than  reported  in  this  paper,  but  was  not 
exhaustive,  and  uncovered  no  erroneous  deci- 
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sions.  However,  one  false  decision  to  link 
or  merge  sets  of  locations  could  result  in  a 
sequence  of  false  decisions  involving  those 
sets.  But,  the  initial  false  decision  intro¬ 
duces  systematic  error  into  the  set  repre¬ 
senting  the  Mergence.  And,  within  the  reso¬ 
lution  of  the  method,  systematic  error  is 
detectable  in  the  residuals  that  result  from 
a  maximum  likelihood  solution  [8],  the 
latitude-longitude  residual  set  with  greatest 
energy  value  being  associated  with  the  data 
set  element  responsible  for  the  systematic 
error. 

Though  the  x^eighted  least  squares  spline 
may  not  be  a  maximum  likelihood  solution 
because  the  errors  in  the  observations  are 
not  always  Gaussian,  the  testing  thus  far 
indicates  that  detection  of  such  a  syste¬ 
matic  error  is  feasible  from  residuals  of  the 
Mergence.  It  is  expected  that  a  study  of 
field  test  data  will  make  possible  an 
acceptance  verification  test. 
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Automatic  correlation  of  intelligence  information  involves  the  evaluation  and  comparison 
of  independent  activity  reports  to  determine  if  they  are  related  to  the  same  item  of  interest. 
The  information  that  is  used  in  this  process  is  normally  extracted  from  well-defined  fields 
within  formatted  messages.  In  addition,  correlators  generally  process  only  one  type  of 
intelligence  information  such  as  FLINT  reports.  This  is  based  on  the  common  characteristics 
of  similar  fields  and  the  assumption  that  data  contained  within  reports  from  different  types 
of  intelligence  sources  is  difficult  to  compare.  Comparison  of  information  from  several 
types  of  intelligence  sources,  however,  can  be  supported  by  a  number  of  related  attributes. 
Therefore,  it  is  possible  to  define  an  automatic  correlator  that  not  only  utilizes  these 
common  attributes  to  relate  the  different  sources  but  one  which  fuses  the  data  to  result 
in  more  information  than  is  provided  by  any  single  source. 


The  concept  for  the  automatic  correlation  of  intelligence  reports  generated  by  multiple 
sources  includes  FLINT  and  imagery  reports,  each  of  which  have  strengths  and  limitations 
which  complement  each  other.  The  FLINT  reports  are  based  primarily  on  the  emanations  of 
radar  systems  which  are  a  major  element  of  many  military  threats.  These  sources  collect 
a  large  volume  of  data  from  emitters  at  world  wide  locations  including  both  land  and  sea 
and  report  the  information  in  the  form  of  timely,  well  defined,  fixed  format  messages.  They 
are  often  limited  in  terms  of  location  accuracy  and  specific  functional  identification  o^ 
the  site  generating  the  emission.  The  emitter  must  be  active  and  detected  to  be  reportea. 
Imagery  reports  are  based  on  interpretation  of  imagery  collected  over  a  target  area.  These 
sources  are  not  dependent  on  the  activity  of  the  site,  as  are  FLINT  systems,  and  provia^ 
more  precise  information  including  location  and  site  functionality  as  well  as  information 
that  is  not  provided  by  FLINT  such  as  site  elevation  and  operational  status.  Imagery  resuj-ts 
are  also  reported  in  the  form  of  well-defined  fixed  format  messages.  Their  two  major  limicdtion 
are  the  relatively  small  georgraphic  area  covered  by  any  single  image  and  the  amount  of  time 
required  to  interpret  the  imagery  and  generate  the  reports. 

These  sources  share  a  common  set  of  attributes  that  serve  as  the  basis  for  the  fusion 
process.  They  include  the  time  of  the  reported  activity,  the  geographic  location  information 
such  as  latitude,  longitude  and  BE  numbers,  as  well  as  site  characteristics  such  as  ELNOT 
and  weapon  type  within  the  FLINT  reports  and  the  equipment  name  within  the  imagery  reports , 

By  relating  these  sources  based  on  the  correlation  of  their  common  attributes  and  by  fusing 
or  combining  the  information,  it  is  possible  to  create  an  automated  system  that  uses  the 
strengths  of  each  source  while  minimizing  their  weaknesses.  FLINT  sources  provide  world 
wide  coverage  and  support  the  near  real-time  identification  of  new  or  previously  unreported 
sites  and  the  tracking  of  mobile  systems.  Imagery  sources  provide  more  complete  information 
as  well  as  the  site  status.  An  automated  multiple  source  correlation  and  fusion  process 
is  directly  applicable  to  any  military  operation  requiring  near  real-time  information  on 
identification,  location.  Order  of  Battle,  and  operational  status  of  military  threats. 


425 


Automatic  Correlation  Of  Multiple 
Intelligence  Sources 


Fred  Schroeder  and  Larry  Wright 


BTG,  Inc. 

1945  Old  Gallows  Road 
Vienna,  Virginia  22180 


INTRODUCTION 

Effective  mission  planning  and 
situation  assessment  require  the  use  of  time 
critical  information  to  characterize  enemy 
disposition  and  defenses  must  be  considered 
during  the  planning  process.  BTG  Inc.  is 
actively  involved  in  the  development  and 
deployment  of  computer  systems  that  receive 
and  correlate  near  real-time  intelligence 
information  and  provide  the  resulting  data  to 
mission  planning  and  situation  assessment 
systems.  Currently,  we  are  involved  in  the 
specification  and  design  of  a  data  fusion 
system  referred  to  as  the  Near  Real-  Time 
Threat  Processor  (NRTTP)  to  be  included  as  an 
upgrade  to  a  mission  planning  center  (MPC) . 
The  MPC  is  responsible  for  route  planning  and 
mission  preparation  for  a  large  number 
targets  each  year  and  is  being  upgraded  to 
utilize  many  automated  techniques.  As  part 
of  the  automated  route  planning  process,  the 
system  will  take  into  consideration  the 
location  and  type  of  threats  maintained  by 
the  NRTTP,  This  paper  discusses  the 
characteristics  of  the  threats,  the  types  and 
sources  for  intelligence  information  used  to 
monitor  their  activities,  the  method  used  to 
correlate  the  individual  sources  and  the 
process  of  fusing  the  multiple  sources  to 
form  a  more  complete  representation  of  the 
threat , 

THREAT  CHARACTERTSTTCg; 

Threats  to  the  MPS  missions  include 
both  afloat  and  land  based  defensive  systems. 
Many  of  these,  including  those  on  land,  are 
mobile  and  can  change  locations  within  short 
periods  of  time  (hours) .  These  threats  can 
be  divided  into  three  major  categories  based 
on  their  degree  of  mobility.  They  are: 

a .  Fixed  sites 

b.  Mobile  with  pre-planned  sites 

c.  Mobile  with  no  pre-planned  sites 

The  fixed  threat  sites  include  some 
types  of  surface  to  air  missile  (SAM) 
facilities  and  air  fields.  Since  these  sites 


don't  move,  they  can  be  studied  over  time  by 
intelligence  organizations  to  determine  their 
location,  functionality  and  operational 
characteristics.  This  information  can  then 
be  provided  to  the  planning  centers  and 
updated  periodically.  However,  even  though  a 
historical  view  of  the  site  can  be 
maintained,  it  is  necessary  to  use  current 
intelligence  data  to  determine  the  site's 
current  status,  particularly  with  regard  to 
battle  damage  assessment. 

The  mobile  threats  with  pre-planned 
sites  include  systems,  such  as  some  types  of 
SAMs,  that  can  be  moved  from  one  location  to 
another  but  which  require  some  type  of 
physical  site  preparation.  In  general,  these 
types  of  threats  have  many  more  prepared 
sites  than  weapon  systems  to  occupy  them. 
These  pre-planned  sites  are  characterized 
over  time  by  intelligence  organizations  but 
the  current  locations  of  the  weapon  systems 
can  only  be  determined  by  the  analysis  of 
current  intelligence  data. 

The  third  type  is  the  mobile  with  no 
pre-planned  sites.  These  include  ships  and 
some  types  of  land  based  systems  including 
ZSU-23  AAA  Guns  and  tactical  SAMs. 
Intelligence  organizations  attempt  to 
characterize  the  general  operational  areas 
for  these  types  of  systems  but  the  current 
locations  can  only  be  determined  by  the 
analysis  of  near-real-time  intelligence 
data . 

Information  related  to  all  three  of 
these  types  of  threats  is  provided  to  the 
mission  planning  organizations  in  the  form  of 
Order  of  Battle  Data  Bases. 

FLINT  CHARACTF.RT.qTT(pff 

Because  of  the  mobility  of  the  threats 
and  the  need  for  current  status  information, 
it  is  necessary  for  the  planning  center  to 
augment  the  historical  threat  information 
with  ^  more  current  data  collected  and 
distributed  by  various  Intelligence  sources. 
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A  primary  source  of  this  type  of  information 
is  electronic  intelligence  (ELINT) . 

ELINT  systems  are  based  primarily  on 
radars  which  are  a  major  characteristic  of 
most  defensive  systems.  Typical  applications 
include  target  acquisition,  tracking  and 
guidance  for  the  weapon  systems  themselves 
and  as  navigation  for  ships .  Many  of  the 
radars  are  unique  to  a  particular  function  or 
even  a  weapon  system.  In  addition,  some  of 
the  weapon  systems  not  characterized  by  a 
single  unique  radar  are  identifiable  based  on 
a  specific  set  of  radars.  ELINT  systems  are 
capable  of  receiving  and  distributing  large 
volumes  of  data  collected  world  wide.  They 
are  often  very  responsive  and  can  distribute 
radar  contact  information  very  quickly. 
Because  of  this,  ELINT  collection  systems  are 
excellent  sources  for  the  detection, 
functional  identification  and  location  of 
threats . 

BASI.q  FOR  NRTTP  ELTNT  CORRELATOR 

ELINT  information  is  used  in  a  number 
of  systems  that  are  currently  deployed  to 
support  planning  activities.  The  two  that 
form  the  basis  for  the  NRTTP  correlator  are 
the  Prototype  Ocean  Surveillance  Terminal 
(POST)  and  the  Prototype  Analysis  Workstation 
(PAWS) .  These  systems  are  based  on  a  common 
set  of  heuristics  implemented  within  an 
expert  system  correlator  that  processes  many 
forms  of  ELINT  data  and  maintains  or  tracks 
the  positions  of  ship  and  land  emitters. 

In  order  to  discuss  the  correlation 
process,  it  is  first  necessary  to  define  and 
briefly  describe  the  characteristics  or 
attributes  reported  by  the  ELINT  sources.  In 
general,  ELINT  information  is  reported  as 
parameter  values  or  distinct  characteristics 
of  emitters  that  serve  to  identify  the 
individual  emitter  and  its  function.  There 
are  two  basic  types  of  ELINT  parameters, 
descriptive  and  measurable.  They  are 
reported  in  textual  messages  that  contain 
fixed  format  fields  that  support  automated 
processing.  An  example  of  the  Tactical  ELINT 
(TACELINT)  message  format  is  shown  in  Figure 
1.  The  top  half  of  this  figure  contains  the 
format  for  the  message  with  the  names  of  the 


fields  in  parentheses,  "{)".  Those  that  are 
used  in  the  correlation  process  are 
underlined.  The  other  values  are  as  they 
would  appear  in  an  actual  message.  The 
bottom  half  contains  the  list  of  fields  that 
are  used  in  the  correlator  and  examples  of 
their  content. 

The  descriptive  parameters  include 
general  characteristics  of  the  signal  that 
was  received,  possibly  resulting  from  other 
types  of  analysis.  They  include: 

a.  ELINT  Notation  (ELNOT) 

b.  Location 

c.  Ellipse  size  and  orientation 

d.  Ship  Name 

e .  NOIC  ID 

The  ELNOT  is  the  type  or  class  of  radar 
assumed  to  have  generated  the  related  signal. 
The  location  and  ellipse  size  and  orientation 
provide  geographic  information.  The  ship 
name  is  often  associated  with  afloat  based 
emitters  and  is  the  platform  from  which  the 
signal  is  thought  to  have  originated.  These 
parameters,  especially  the  ELNOT  and 
geographic  information,  are  used  within  the 
correlator . 

The  correlator  is  primarily  based  on 
the  measurable  parameters  associated  with  the 
ELINT  information.  These  include: 

a.  Radio  Frequency  (RF) 

b.  Pulse  Duration  (PD)  or  Pulse  Width 

(PW) 

c .  Beamwidth  (BW) 

d.  Pulse  Repetition  Interval  (PRI)  or 

Pulse  Repetition  Frequency  (PRF) 

e. Scan  Rate  (SR)  or  Interval 

The  RF  is  the  transmitted  frequency  of 
the  radar  and  can  be  used  to  support  the 
determination  of  its  use  and  physical  size. 

The  PD  or  PW  is  the  time  that  the  radar 
is  radiating  energy  and  can  be  used  to 
determine  the  minimum  range  of  the  radar  and 
its  range  resolution  (the  ability  to  identify 
multiple  targets  close  together) . 


EXAMPLE  TACTICAL  ELINT  (TACELINT)  REPORT  FORMAT 

UNOAS 

MSG1D/TACELINT/(0RGNTR)/(MSG  SER  NO)/(MONTH)/(QUALIFIER)/(QUAL  SER  NO)// 

COLLINFO/(COLL  DIGRPH)/(COLL  TRIGRPH)/(COLL  MSN  NO)/(COLL  PRJ  NAME)// 

SOI/(TGT  SGNL  IDVfDTCT  T1  MW/TIM  LOS'n//ELNOTV/EMTR  DSG)/(CTRY  OF  EVT)/fTRG  IDV(UNITWfWEAPVfEMIT  FNC)// 
PRM/(ENTRY)/iEEl/(RF  OPR  MODE)/(PRI-PRF)/(PRI  ACTVTY  COD E)/(EDl/( SCAN  TYPE)/rSCAN  RATEWfPOLRZTNl// 


FieWs  an>J  Example  Valygs 


DTCTTIM 

06081 2Z 

ORIENTATION  - 

027.5T 

ELNOT 

A101A 

SEMI-MAJ 

02.4NM 

•me  ID 

P00418002 

SEMI-MIN 

01.8NM 

WEAR 

SA-1 

RF 

827MHZ 

l£C  DATA  CAT  - 

F 

PRI-PRF 

PRI:002015.3 

UDC 

LS:512242N0115030E 

PD 

PD:3.200 

RADIUS 

03.7NM 

SCANRA7E 

4.2SPC 

Figure  1. 
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The  BW  is  a  measure  of  the  area 
coverage  for  the  radar  and  is  usually 
measured  horizontally  and  vertically  in 
degrees  for  the  main  beam  or  lobe.  This 
effects  the  probability  of  the  radar 
detecting  an  object  and  the  accuracy  of  the 
elevation  and  azimuth  measurements . 

The  PRI  and  PRF  are  inverses  of  one- 
another  and  measure  the  amount  of  time 
between  radar  pulses.  This  parameter  is  the 
bases  for  computing  the  Maximum  Unambiguous 
Range  which  also  is  related  to  its 
operational  use.  Due  to  its  consistency  and 
unique  values  for  a  given  radar,  especially 
for  those  that  are  crystal  controlled,  it  is 
one  of  the  best  correlation  parameters. 

The  last  parameter  is  the  SR  or 
Interval  which  is  how  often  a  target  is 
illuminated  by  the  beam  of  energy.  This  is 
dependent  on  type  of  radar  and  its  function 
and  is  relevant  for  most  types  of  radars. 
Again,  due  to  its  consistency  and  unique 
values  for  a  given  radar,  it  can  be  a  very 
good  correlation  parameter  and  in  some  cases 
better  than  the  PRI  or  PRF. 


ELINT  CORRELATION 

For  the  purposes  of  correlation,  the 
fields  contained  in  the  FLINT  reports  have 
been  divided  into  three  types .  The  first 
type  is  geographic,  which  consists  of  the 
position  information  such  as  latitude  and 
longitude,  and  ellipse  size  and  orientation. 
The  second  type  is  the  non-parametric 
attribute,  which  consists  of  description 
information  such  as  the  ELNOT,  Ship  Name,  and 
BEN. 

The  third  type  is  the  ELINT  parametric 
information  which  consists  of  the  PRI,  PRF, 
RF,  SR,  PD,  and  PW. 

Based  on  these  three  types  of  data 
(geographic,  non-parametric  attribute,  and 
ELINT  parametric)  the  correlator  performs 
three  types  of  processing.  At  each  step  the 
candidate  contact  reports  are  compared  to  the 
current  contact  being  correlated  and  a  score 
computed.  The  score  represents  a 

quantitative  measure  of  how  well  the  contacts 
match.  The  following  is  an  overview  of  the 
type  of  processing  preformed  for  each  step  in 
the  correlation  process. 

The  first  step  is  geographic 
correlation  which  evaluates  the  previously 
reported  contacts  to  identify  a  set  of 
candidates  that  are  geographically  feasible. 
This  is  based  on  the  geographic  proximity, 
size  and  orientation  of  the  error  ellipses 
being  compared,  required  speed  to  reach  the 
new  location,  and  for  ships,  the  required 
course  change.  This  results  in  a  geographic 
score  for  each  candidate. 

The  second  step  in  the  correlation 
process  is  based  on  the  non-parametric 
attributes  and  is  referred  to  as  associative 
correlation.  This  process  involves  the 
comparison  of  the  attribute  values  to 
determine  if  they  are  related  to  the  same 
emitter.  It  is  based  on  the  number  of 
attributes  provided  in  the  current  and 
candidate  report,  and  the  number  of  matches. 
This  results  in  an  attribute  score  for  each 
candidate. 


The  third  and  primary  step  in  the 
correlation  process  is  based  on  the  ELINT 
parameters.  This  is  also  a  multi-step 
process  that  compares  the  ELINT  parameters 
provided  in  the  contact  report  with  those  of 
previous  reports.  The  first  step  is  to 
convert  the  PRI  or  PRF  to  a  base-band  value. 
This  is  particularly  useful  for  radars  that 
vary  the  the  pulse  interval  or  "count  down" 
value  based  on  a  crystal  oscillation  rate. 
The  base-band  value  then  becomes  a  single 
value  to  be  used  in  the  comparison  process. 
Some  of  these  count  down  values  can  be  found 
in  Electronic  Parameter  Lists  (EPLs)  .  The 
correlator  utilizes  an  emitter  data  base  that 
contains  the  base-band  PRI  range  for  each 
class  of  emitter  processed  by  the  correlator. 
(Note:  If  an  ELINT  contact  refers  to  an 
emitter  that  is  not  included  in  the  emitter 
data  base  it  is  still  correlated  and  tracked 
however,  its  description  is  not  augmented  by 
the  additional  data  contained  in  the  data 
base  and  its  functional  and  operational 
characteristics  must  be  approximated.)  In 
cases  where  a  given  emitter  class  has 
slightly  different  measurable 
characteristics,  the  class  is  divided  into 
versions  based  on  the  PRI  ranges. 

Once  the  base-band  PRI  is  determined, 
an  ELINT  score  is  computed  based  on  how  well 
the  PRI,  SR,  PW  and  RF  of  the  current  contact 
match  the  candidate  values.  The  scoring 
includes  the  use  of  a  parameter  weight 
obtained  from  the  emitter  data  base  for  each 
of  the  four  ELINT  parameters.  (Note:  If  the 
candidate  is  not  in  the  emitter  data  base,  a 
set  of  default  values  is  used.) 

An  overall  total  score  is  then 
generated  by  merging  the  geographic, 
attribute  and  ELINT  scores  and  selecting  a 
best  correlation  candidate.  If  only  one 
candidate  is  found,  the  current  contact 
report  is  merged  to  that  track  or  site.  If 
no  candidates  are  found,  a  new  track  or  site 
is  created  based  on  the  current  report.  If 
more  than  one  candidate  is  found,  a  complex 
rule  based  three  (3)  phase  (geographic,  non¬ 
parameter  attribute  and  parameters)  multi¬ 
pass  process  is  performed  which  evaluates  the 
individual  scores  singly  and  in  combination 
with  all  three  factors,  a  process  from  which 
the  single  best  candidate  is  identified. 

TEMPLATE  MATCHING 

When  a  new  track  or  site  is  created  for 
land  based  emitters,  a  special  type  of 
processing  is  performed  to  attempt  to 
determine  if  the  emitter  is  one  of  several 
emitters  associated  with  a  single  operational 
element.  Examples  of  this  t3(Tpe  of  defensive 
system  include  SAMs  that  utilize  several 
different  types  of  radars  such  as  those  used 
for  target  acquisition,  tracking  and 
guidance.  This  process  is  referred  to  as 
template  matching  and  utilizes  a  data  base 
that  describes  the  relationships  of  emitters 
used  by  different  types  of  operational 
elements.  The  template  matching  process 
compares  the  new  emitter  and  those  in  its 
geographic  proximity  with  the  templates 
identified  in  the  data  base.  If  there  is  no 
match,  the  emitter  continues  to  be  tracked  as 
an  individual.  If  there  is  a  match,  the 
emitter  is  arouoed  with  the  other  related 
emitters  and  tracked  as  a  group.  This  greatly 
increase  the  amount  of  information  known  about 
the  defensive  system  by  identifying  the 
functional  characteristics  of  the  site  (such 
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as  a  SAM  site)  as  opposed  to  only  knowing  the 
use  of  the  specific  radar  (such  as  range 
finding) .  It  also  increases  the  potential  for 
proper  tracking  since  an  update  to  any  of  the 
individual  emitter  is  also  an  update  to  the 
group  and  can  be  used  to  refine  the  sites 
actual  position. 

ELINT  LIMITATIONS 

ELINT  sources  are  very  responsive  and 
provide  a  large  amount  of  information, 
however,  they  are  limited  in  several  areas.  A 
single  ELINT  report  has  limited  location 
accuracy  which  is  normally  reported  as  an 
error  ellipse  that  is  measured  in  miles.  This 
accuracy  can  be  improved  as  multiple  reports 
are  combined  and  the  overlap  of  the  ellipses 
are  evaluated.  This  process  is  shown 
graphically  in  Figure  2.  This  however, 
requires  more  time  to  collect  the  additional 
information.  In  addition,  these  reports  do 
not  contain  certain  types  of  data  that  are 
needed  during  the  planning  process.  An 
example  is  elevation  data .  This  is  ir[^>ortant 
to  the  precise  positioning  of  the  threat  and 
therefore,  the  determination  of  its  geographic 
operational  constraints.  For  example,  the 
threat  may  be  located  at  the  top  of  a  hill  or 
within  a  valley  contained  within  the  same 
error  ellipse. 

Probably  the  most  limiting  aspect  of  the 
ELINT  sources  is  that  they  only  report 
positive  information  (the  contact  of  an 
emitters  pulse)  and  can  not  report  the  absence 
of  an  emitter.  This  results  in  inactive 
emitters  not  being  reported  and  emitters  that 
have  moved  and  not  reported,  to  continue  to  be 
associated  with  their  old  location.  (Note; 
For  sea  based  emitters,  an  estdLmate  of  the 
current  location  can  be  made  base  on  the 
historical  course  and  speed.) 


ATHF.R  SOTTRr.ER  OF  INTELLIGENCE  INFORMATION 

ELINT  represents  only  one  of  many  types 
of  intelligence  information  available  to  the 
mission  planning  or  situation  assessment 
organizations.  Others  include  Photographic 
Intelligence  (PHOTINT) ,  Imagery  Intelligence 
(IMINT),  Communication  Intelligence  (COMINT), 
Radar  Intelligence  (RADINT) ,  Measurement  and 
Signature  Intelligence  (MASINT) ,  Visual 
Intelligence  (VISINT) ,  Telemetry  Intelligence 
(TELINT),  Acoustic  Intelligence  (ACINT) ,  and 
Human  Intelligence  Sources  (HUMINT) . 

These  sources  provide  very  different 
types  of  information  in  many  formats  and  have 
very  different  response  times.  Of  these,  a 
major  source  of  land  based  information  is 
imagery.  This  information  is  reported  in  the 
form  of  a  photograph  or  image  and  provided  to 
human  interpreters  for  evaluation.  The 
imagery  itself  has  little  use  in  the 
automated  correlation  process.  However,  the 
result  of  interpreter's  evaluation  is 
normally  a  report  that  describes  the  content 
of  the  image .  These  reports  include  fixed 
format  fields  that  are  very  suitable  for 
automated  processing. 

IMAGERY  REPORT  CHARACTERISTICS 

Imagery  collections  systems  are  able  to 
provide  more  complete  information  related  to 
the  defensive  systems.  This  often  includes  a 
more  precise  description  of  the  type  of 
threat,  its  location  and  addition  physical 
and  operational  characteristics  and  is 
provided  to  mission  planning  organizations  in 
the  form  of  imagery  interpretation  reports. 
These  reports  do  not  contain  the  actual 
photographs  but  instead  contain  the  results 
of  the  intelligence  analysts  evaluation  of 
the  imagery  and  often  are  available  within 


CONVOLVING  ELLIPSES 


•  Each  ellipse  provides  emitter 
location  Information. 

•  The  convolving  process  combines 
the  ellipse  data  based  on  past 
and  present  contacts  to  allow 
for  a  more  accurate  location 
estimation. 


Figure  2 . 
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hours  of  the  receipt  of  the  imagery.  An 
example  Imagery  Interpretation  Report  (HR) 
is  shown  in  Figure  3.  The  top  half  of  this 
figure  contains  the  format  for  the  message 
with  the  names  of  the  fields  in  parenthesis, 
"0".  Those  that  are  used  in  the  correlation 
process  are  underlined.  The  other  values  are 
as  they  would  appear  in  an  actual  message. 
The  bottom  half  contains  the  list  of  fields 
that  are  used  in  the  correlator  and  exarrples 
of  their  content.  These  types  of  reports 
typically  include  the  following  types  of 
data: 

a .  Time  On  Target 

b.  Geographic  Location 

c .  Elevation  Of  Target 

d.  Target  Name 

e .  Object  Count 

f .  Target  Category  Code 

g.  Order  of  Battle 

h.  Order  of  Battle  Equipment  Name 

i.  Basic  Encyclopedia  Number 

j .  Operational  Status 

k .  Validity 


The  Time  On  Target  and  Geographic 
Location  are  basic  information  related  to 
when  the  image  was  taken  and  the  location  of 
the  object  being  reported.  The  Elevation  Of 
Target  is  included  for  new  sitings  and 
represents  a  type  of  information  .that  is 
difficult  to  obtain  from  other  sources.  The 
Target  Name  is  determined  based  on  the 
interpreters  evaluation  of  the  object  and  is 
related  to  the  Basic  Encyclopedia  Number 
(BEN).  The  BEN  identifies  the  specific  site. 
The  Target  Category  Code  contains  an  entry 
from  the  Intelligence  Data  Handling  System 
(IDHS)  and  is  also  related  to  the  BEN.  The 
Order  of  Battle  (OB)  contains  an  entry  from  a 
list  of  ten  (10)  possible  types.  The  Order 
of  Battle  Equipment  Name  contains  the 
standard  OB  type  or  NATO  code.  If  there  are 
multiple  objects  of  the  same  type,  the  report 
normally  includes  an  Object  Count . 


Two  major  components  of  the  information 
provided  by  the  imagery  interpretation 
reports  are  the  site  status  and  the  degree  of 
certainty  or  validity  associated  with  the 
reported  information.  The  Operational  Status 
provides  both  threat  activity  and  inactivity 
information.  It  contains  one  of  over  twenty 
different  status  types  which  range  from 
general  to  very  specific.  Examples  include 
"operational"  and  "not  operational"  as 
general  status,  "occupied"  or  "unoccupied"  in 
relation  to  pre-planned  sites,  and  "diiromy"  in 
relation  to  radar  sites  used  only  for 
deception. 

IMAGERY  REPORT  LIMITATIONS 

Imaging  systems  also  have  a  number  of 
limitations.  In  general,  they  produce  a 
smaller  volume  of  data  and  require  more  time 
to  process  than  ELINT  sources  and  are 
normally  less  timely.  In  addition,  they  are 
subject  to  errors  by  the  interpreter  and 
include  general  types  of  information  such  as 
construction  activity  that  are  not  related  to 
a  threat  or  threat  status . 

FUSING  QF_ELINT  AND  IMAGERY.  REPORTS 

The  combination  of  these  two  types  of 
data,  ELINT  and  imagery  reports,  can  provide 
a  very  complete  and  accurate  description  of 
the  type,  location,  and  status  of  threats. 
The  ELINT  information  can  be  used  for  both 
land  and  sea  based  defensive  systems  and 
provides  world  wide  area  coverage .  It  is  the 
major  source  of  information  used  to  identify 
new  threats  not  associated  with  fixed  or  pre¬ 
planned  sites.  This  includes  ships  near 
shore  and  fully  mobile  SAMs.  In  addition,  it 
can  be  used  to  identify  active  pre-planned 
sites  and  to  track  the  movement  of  other 
mobile  systems.  The  imagery  report 
information  is  used  to  provide  more  accurate 
and  complete  information  for  land  based 
threats .  This  includes  location,  elevation, 
and  equipment  type.  In  addition,  it  is  used 


EXAMPLE  IMAGERY  INTERPRETATION  REPORT  (HR)  FORMAT 

UNCLAS 

MSGtD/IIR/(ORGNTR)/{MSG  SER  NO)/{MONTH)/(QUALIFIER)/(QUAL  SER  NO)// 

RPTID/(RPT  INDIC:EG.,  1P1R)/(RPTNG  ORG)/(RPT  SER  NO)/(PRJ  ID)/(IMG  MSN  NO)/(MSN  DATE)// 

HIUTES/(MiSSION  HIGHLIGHTS)// 

ITEMTYPE/(TYPE  OF  ITEM  RPTD)// 

ITEM/(1TEM  NO)/(TARG  NAMEVIBE  NUMBERV/TARG  CA-n/(CTRV  OF  EVENT)/(NAT  TSKNG  INDIC)// 

LOC/fGEO  COORDSVILJTM  COORnsU/ 

OTID/(TGT  LGTH)/(TGT  WDTH)/{TGT  A2MTH)/(TAR(3  ELEV)/(COMIREX  REQ  NO)/(MIL  RGN  CODE)// 

STATACT/(QP  STAT)/(SCNDRY  STATUS)/(CHG  SiGNlF)/{PRJ  ID)/(IMG  MSN  NO)/(MSN  DATE)/(AUTO  TAG  TARG  GRPHC)// 
RMK/(TGT-AC7VTY  REMARKS)// 

OBID/(OS  TYPE)/(DAT  OF  CNV)/a.!M  ON  TARg)/(ATTG-NBRG  CMNT)/(CHG  SIGNIF)/(PRJ  ID)/(IMG  MSN  NO)/(MSN  DATE)// 
OBEQ/fQB  CQUNTl/fVAI.IDITY^/IOB  EQ  NAME^/(OBJ  CODE-FUTURE  USE)/(OB  EQUIP  CMNTS)// 


TARG  NAME 
BE  NUMBER 
TARG  CAT 
CeDOOCRDS 
UTM  COORDS 
TARGELEV 
OPSTAT 


Fields  Used  and  Example  Values 


WESTERN  ARMY  BARRACKS 
BEN:0235-00235 
CAT:90000 
GEO;221 600N1 05301 2E 
UTM;1 8STN408898 
ELE:03580M 
OPR 


06  TYPE 
DATOFCNV 
TIM  ON  TARG 
OB  COUNT 
VALIDITY 
OB  EQ  NAME 


MIS 

840602 

TOT:1207Z 

0103 

OOKF 

SA-6 


Figure  3 . 
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to  identify  new  sites  not  detected  by  the 
ELINT  collection  systems.  Most  importantly, 
it  is  used  as  a  source  of  negative 
information  such  as  a  pre-planned  site  not 
currently  occupied  by  its  related  weapon 
system  or  a  fixed  site  not  currently  in 
operation  due  to  maintenance  activities . 

This  fusion  process  is  based  on  the 
correlation  of  a  common  set  of  attributes. 
These  include  general  types  of  information 
such  as  the  time  of  the  reported  activity, 
the  geographic  location  information  such  as 
latitude  and  longitude  and  BEN.  It  also 
includes  specific  site  information  such  as 
the  ELNOT  and  weapon  type  within  the  ELINT 
reports  and  the  equipment  name  within  the 
imagery  reports.  Figure  4  shows  the 
relationship  of  the  fields  contained  in  the 
TACELINT  and  HR  reports,  both  to  themselves 
and  to  each  other.  As  shown,  the  left  coltimn 
shows  the  fields  used  to  correlate  ELINT 
reports.  The  first  set  of  fields  in  the 
column  is  common  to  both  TACELINT  and  SELOR 
RED  (another  ELINT  message  format) .  The 
second  and  third  sets  of  fields  are  specific 
to  each  message  format .  The  second  column  is 
the  same  as  the  first  and  the  lines  between 
them  show  which  fields  are  related.  The 
third  and  fourth  colxamns  are  for  HR  messages 
and  the  lines  between  them  show  their 
relationships.  The  final  set  of  lines 
between  the  ELINT  and  HR  fields  shows  the 
interrelationships  between  the  two  sources. 

The  Target  Id  of  the  ELINT  report 
contains  either  a  Placename  Identification 
Number  (PIN)  or  a  BEN.  The  PIN  when  combined 
with  the  geographic  location,  can  also  be 
translated  to  a  BEN.  This  commonly  defined 
value  can  then  be  used  to  relate  the  TACELINT 
Target  Id  to  the  BEN  of  the  HR. 


The  Order  of  Battle  Equipment  Name  of 
the  HR  reports  the  specific  type  of 
equipment  seen  in  the  image  based  on  the  IDHS 
equipment  list.  The  Weapon  Type  of  the  ELINT 
report  is  selected  from  a  weapons  list. 
Based  on  the  analysis  of  the  list  of  possible 
field  values  contained  in  the  related 
instruction  manuals  used  to  prepare  these 
reports,  it  is  possible  to  generate  a 
translation  table  that  converts  the  field 
values  of  one  report  to  those  of  the  other. 
This  translation  table  then  defines  the 
relationship  between  the  two  reports. 


The  NRTTP  refers  to  this  translation 
table  as  the  Attribute  Data  Base.  This  data 
base  also  contains  supporting  information 
that  augments  the  content  to  the  fields.  An 
example  of  how  this  translation  process  is 
performed  is  shown  in  Figure  5 .  This  example 
shows  a  subset  of  the  fields  from  an  HR  and 
TACELINT  report .  The  Attribute  Data  Base  is 
used  to  translate  the  SAM-6  equipment  name  of 
the  HR  and  the  SA-6  weapon  type  of  the 
TACELINT  to  the  common  correlator  value  of 
SA6.  In  addition,  the  values  are  augmented 
by  their  respective  OB  types.  (Note:  Due  to 
similarities  in  values,  in  some  cases  it  may 
be  possible  to  develop  a  translation 
algorithm  that  does  not  require  every 
possible  field  value  to  be  pre-defined  in  the 
data  base.  Our  design  currently  identifies  a 
generic  table  which  could  be  replaced  or 
augmented  by  a  process  of  this  type  at  a 
later  time . ) 

The  Order  of  Battle  Equipment  Name  of 
the  HR  report  is  also  related  to  the  ELNOT 
of  the  TACELINT  report.  This  occurs  when  the 
HR  references  a  radar  system  by  name  which 
can  then  be  associated  with  its  ELNOT  by  the 
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correlator.  This  then  allows  the  HR  to  be 
indirectly  related  to  the  ELNOT  of  the 
TACELINT  report. 

Given  the  relationships  described  above 
the  fusion  process  is  very  similar  to  that 
performed  by  the  ELINT  correlator.  The 
fields  of  the  two  types  of  reports  are 
divided  into  three  categories.  These  are 
geographic,  attribute  and  emitter.  While  the 
ELINT  reports  normally  contain  all  three 
types  of  data,  the  HR  fields  are  primarily 
categorized  as  geographic  and  attribute.  In 
special  cases  as  describe  above,  it  is 
possible  for  the  HR  equipment  name  to  be 
converted  to  an  ELNOT  which  results  in  it 
being  categorized  as  an  ELINT  field.  Once 
the  fields  have  been  categorized,  the 
geographic  and  attribute  values  are 
translated  into  a  common  set  as-  described 
above.  This  includes  converting  the 
geographic  ELINT  PIN  into  a  BEN  and 
translating  the  attributes  into  a  common 
nomenclature.  The  correlator  then  generates 
an  attribute,  ELINT  and  geographic  score  as 
done  for  ELINT  correlation.  These  are  then 
used  to  select  candidate  matches  and  finally 
to  determine  the  disposition  of  the  contact 
report  being  processed.  Once  the  correlation 
has  been  performed,  the  related  track  or  site 
entry  is  updated  with  the  new  information.  In 
the  case  of  an  imagery  report,  the  entry  is 
also  updated  with  its  status. 

SUMMARY 

Automatic  correlation  and  fusion  of 
ELINT  and  imagery  reports  will  provide  the  MPC 
with  a  very  complete  and  timely  location  and 
operational  status  information  related  to  the 
defensive  systems  that  are  threats.  Due  to 
its  general  characteristics,  this  capability 
is  applicable  to  all  mission  planning 
activities  that  require  time  critical 
information.  In  addition,  it  is  possible  to 
include  other  types  of  intelligence  data  by 
identifying  the  relationships  between  the 
reported  information  and  expanding  the  set  of 
rules  for  fusing  them,  as  was  done  for  the 
TACELINT  and  HR  reports. 


EXAMPLE  USE  OF  ATTRIBUTE  DATA  BASE 


Report/  Order  of 

Field  Name  Identifier  Translation  Battle 

I  HR/OBEQNAME  |  SAM-6  |  SA6  |  MOB  | 


Figure  5 . 
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INTRODUCTION 

One  of  the  functions  of  a  data  fusion  system  involves 
identifying  the  platforms  reported  in  Electronic  Intelligence 
(Elint)  data.  These  reports  contain  a  set  of  parameters  measured 
from  the  platform's  radar.  The  assignment  of  a  name  to  this 
radar  report  is  called  Hull-to-Emitter  Correlation  (hultQc). 

In  [4]  the  authors  described  an  Adaptive  Network  System 
(ANS)  (popularly  refered  to  as  "neural  network")  approach  to 
this  problem.  The  particular  paradigm  used  was  a 
backpropagation  network.  This  system  classifies  the  data  into 
linearly  separable  classes,  using  a  gradient  descent,  least  mean 
squared  error  algorithm  to  determine  the  separating  hyperplanes. 
The  network  returns  a  value  for  each  hull  that  gives  a  measure  of 
its  distance  from  a  point  in  the  teaching  set  (actually  it’s  a 
measure  of  the  distance  from  the  separating  hyperplane).  ^  This 
approach  compared  favorably  to  traditional  statistical  techniques. 
One  problem  with  this  approach  is  that  the  output  of  the 
network,  though  it  ranges  continuously  from  zero  to  one,  has  no 
relationship  to  probability.  Although  the  network  gives  a 
ranking  for  the  hulls  it  has  learned,  the  value  associated  with  any 
one  hull  gives  little  information.  It  is  only  the  value  relative  to 
the  other  hulls  that  can  be  used  in  further  decision  making. 

This  paper  deals  with  a  modification  of  a  statistical  technique 
that  retains  the  distributed  nature  of  the  ANS,  while  allowing  the 
network  to  learn  to  approximate  the  probability  distribution  of 
the  data.  This  not  only  allows  the  network  the  potential  of  better 
performance,  but  it  allows  other  systems  to  use  the  output  of  the 
network  in  a  probablistic  sense. 

THE  HULTEC  PROBLEM 

For  the  purposes  of  this  paper,  a  report  is  a  set  of  parameters 
measured  from  a  radar  signal:  PRI  (pulse  repetition  interval) j 
Scan  (scan  rate),  RF  (frequency),  etc.  The  hultec  problem  is  to 
determine  the  emitter  of  origin  for  the  report,  based  on  these 
measured  parameters.  By  its  nature,  this  is  a  difficult  problem. 
Errors  in  measurement  are  bound  to  occur,  so  the  system 
requires  a  measure  of  fault  tolerance.  Some  sensors  report 
different  parameters  than  others,  requiring  the  system  to  be  able 
to  handle  missing  data.  The  parameter  spaces  of  several  enutters 
may  overlap,  so  the  system  must  respond  with  a  ranked  list  of 
emitters,  radier  than  a  single  emitter. 

The  main  problem  with  the  backpropagation  approach 
investigated  in  [4]  is  that  it  attempts  to  assign  a  single  hull  to  the 
report.  Often,  as  mentioned  above,  the  distributions  of  ^e  hulls 
are  not  disjoint.  It  is  therefor  not  always  possible  or  desirable  to 
assign  a  report  to  a  single  hull.  Instead,  one  would  like  to 
provide  a  list  of  the  possible  hulls  with  some  measure  of  the 
liklihood  of  classification  associated  with  each  hull.  One  way  to 


do  this  is  to  model  the  probability  density  function  associated 
with  each  hull.  In  other  words,  the  reports  define  a  sample 
space  for  each  of  the  classes  (hulls),  and  the  reports  associated 
with  each  class  define  a  probability  distribution  for  that  class 
over  the  sample  space. 

An  improvement  over  the  ANS  approach  described  in  [4]  is 
one  which  can  learn  the  probability  density  functions  for  the 
classes.  This  is  the  approach  taken  by  the  network  described 
below.  The  output  from  the  net  is  the  value  of  the  probability 
density  functions  for  all  the  hulls  evaluated  on  the  report.  The 
value  output  for  a  given  hull,  in  contrast  to  that  given  by  the 
backpropagation  scheme,  has  a  meaning  in  an  absolute  sense  as 
well  as  relative  to  the  values  output  by  other  hulls.  The  network 
can  indicate  how  certain  it  is  that  a  given  input  stimulus  belongs 
to  a  given  class,  thereby  indicating,  for  example,  that  the  second 
place  choice  for  classification  is  very  high  indeed  and  should  not 
be  disregarded.  This  gives  more  information  than  a 
winner-take-all  approach,  and  this  information  could  be  used  in 
conjunction  with  other  information  (such  as  a  priori  information, 
data  from  other  sensors,  etc.)  to  improve  overall  performance. 

Another  important  advantage  that  a  probability  density 
estimator  gives  over  a  winner- take- all  classifier  is  the  ability  to 
detect  unknown  emitters.  When  the  output  is  low  for  all  the 
emitters  known  by  the  system,  it  is  likely  (though  not  certain) 
that  the  report  comes  from  an  emitter  that  is  unknown  to  the 
system.  The  ability  to  recopize  new  emitters  can  be  as 
important  as  the  correct  classification  of  known  emitters,  and  a 
system  that  can  continue  to  cluster  reports  from  this  new  emitter 
is  of  great  value  to  a  data  fusion  system. 

KERNEL  ESTIMATORS 

The  problem  of  probability  density  estimation  has  been 
attacked  by  many  different  methods.  One  of  the  more  successful 
is  the  technique  of  kernel  estimation  [5].  Consider  a  one 
dimensional  problem,  with  a  single  class.  The  goal  is  to 
aproximate  the  probability  density  function  for  the  teaching  data, 
which  consists  of  a  collection  of  points  drawn  from  an  unknown 
distribution.  In  its  simplest  form,  the  kernel  estimate  involves 
assigning  to  each  of  the  teaching  points  a  gaussian  distribution, 
the  kernel,  with  mean  equal  to  the  input  point  and  a  fixed 
variance,  called  the  window  width.  This  produces,  as  a 
partitioning  of  the  input  space,  a  Voronoi  diagram  [1],  the 
optimal  nearest  neighbor  partitioning.  These  gaussians  are 
summed,  weighted  by  the  window  width  and  the  number  of 
teaching  points.  Therefor,  the  kernel  estimator  with  kernel  K  is 
defined  by 

1  x-Xi 

f(x)  =  --SK( - )  (1) 

nh  h 
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where  h  is  the  window  width.  Distributions  other  than  the 
normal  may  be  used  for  the  kernel  K,  but  for  the  purposes  of 
this  paper,  the  kernel  will  always  be  a  normal  distribution. 

The  ANS  architecture  proposed,  an  implementation  of  the 
kernel  estimator,  assigns  a  node  in  the  hidden  layer  for  each 
gaussian  (see  Figure  1).  The  node  is  connected  to  the  input 
layer,  with  the  weights  set  to  the  mean  of  the  gaussian.  These 
connections  do  a  subtraction,  rather  than  the  traditional 
multiplication  of  their  signal,  and  this  signal  is  then  summed  by 
the  node  and  the  kernel  is  applied.  In  the  case  of  a  single 
dimensional  input  the  node  retains  a  variance,  while  in  the 
multidimensional  case,  the  node  may  retain  a  covariance  matrix, 
or  it  may  keep  a  single  variance. 


INPUT  LAYER  HIDDEN  LAYER 


GAUSSIAN 

CLASSIFICATION 

V _ ) 

FIGURE  1 


Representation  of  Gaussians 


The  gaussians  are  presented  with  the  n-dimensional  input 
from  the  input  layer  having  n  nodes  and  independently  compute 
their  respective  activation  values  (gaussian  distances)  from  this 
input.  The  activation  value  for  these  gaussian  nodes  in  the 
hidden  layer  can  be  obtained  via  the  following  formula: 

exp(-0.5[(s-u)t.rl.(2i-u)]) 

Gj(x) -  (2) 

(271)W/2)  *  ia(l/2) 

Here  Gj(20  is  the  activation  value  of  the  gaussian  node  Gj 
when  presented  with  vector  input  X.  is  the  covariance  matrix 
for  gaussian  node  Gj,  while  is  the  vector-valued  mean  for 
this  gaussian.  d  is  the  dimensionality  of  this  particular  gaussian 
and  is  equated  with  the  dimensionality  of  the  input  layer,  i.e.  the 
number  of  parameters  in  the  report.  Since  the  components  of  the 

mean  are  represented  as  a  node's  input  weights,  these 
weights  can  be  thought  of  as  shifting  the  origin  for  the  node.  A 
gaussian  function  is  then  applied  to  the  inputs  of  the  node. 

If  the  matrix  Z  is  diagonal,  then  (2)  is  just  the  product  of  d 
independant  gaussians,  one  for  each  input.  This  observation 
gives  a  method  for  coping  with  missing  data:  "drop"  the 
appropriate  gaussian  from  the  product.  In  (2)  this  corresponds 
to  assigning  the  missing  input  the  value  at  the  mean  for  that 
component.  The  system  will  continue  to  work  as  if  the  data 
were  there.  This  approach  gives  an  overly  optimistic  output  for 
the  report,  but  the  ranking  will  be  reasonable. 


FIGURE  2 


Combination  of  Gaussians 

The  outputs  from  the  hidden  layer  are  summed,  weighted  by 
the  number  of  points  in  the  node,  to  produce  a  final  answer  (see 
figure  2).  There  is  one  node  on  the  output  layer  per  class.  If 
there  is  more  than  one  class  stored  within  the  network,  the  nodes 
on  the  final  layer  are  connected  only  to  the  nodes  corresponding 
to  their  class.  This  method  of  combining  gaussians  to  produce 
an  overall  distribution  provides  the  system  with  enormous 
flexibility.  Although  the  initial  kernels  being  used  are 
predetermined  to  be  gaussians,  many  widely  varying 
distributions  can  be  approximated  by  a  sum  of  gaussians  (see 
Figure  3).  The  overall  distribution  for  a  particular  class  is  not, 
therefor,  assumed  to  be  normal.  In  this  manner  the  network  can 
be  used  as  a  classifier,  giving  output  that  is  meaningful  in  a 
probabilistic  sense. 


Sum  of  Gaussians 


One  difficulty  with  the  kernel  estimator  is  the  choice  of 
window  width.  A  width  that  is  too  wide  will  produce  a 
unimodal  approximation,  while  a  narrow  choice  can  produce  a 
multimodal  distribution  with  inherent  spikes.  The  width  can  be 
chosen  experimentally,  or  one  of  the  many  adaptive  kernel 
estimation  algorithms  may  be  employed  (see  [5]).  A 
modification  of  the  kernel  estimator  which  allows  the  window  to 
adapt  to  the  data  is  described  below. 
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MOVING  MEAN  AND  VARTANCE 


j*Md  +  X 


(7) 


The  kernel  estimator  described  above  is  static  once  the  initial 
learning  is  finished.  One  way  to  modify  this  scheme  to  produce 
a  dynamic  system  is  to  allow  the  nodes  to  vary  their  means  and 
covariance  matrix  as  the  data  is  recieved. 

Only  those  nodes  whose  activation  values  reach  some 
threshold  defined  by  that  node's  activation  function  (usually  the 
value  of  the  gaussian  at  a  fixed  multiple  of  one  standard 
deviation  from  the  mean)  are  considered  to  be  a  likely  category 
for  the  current  input  and  are  updated.  This  updating  consists  of 
moving  the  mean  and  variance  of  the  gaussian  based  on  the 
current  input  and  some  measure  of  the  total  number  of  inputs  to 
the  gaussian  thus  far.  For  the  updating  of  the  mean,  we  have 

+  [i/a+i)][ia+i)  -  m  o) 

where  p.(j)  is  the  (one-dimensional)  mean  after  the  input  and 

I(j-»-l)  is  the  input  to  be  categorized  in  this  gaussian.  The 
updating  of  the  covariance  is  similar.  Here 

Sxy(j  +  1)  =  Sxy(j)  +  U/G+1)]  *  AxyG+1)  (4) 

with 


l^new  “  . . 

j  =  j  +  1 

Here  j  is  the  number  of  points,  and  x  is  the  input.  In  the  case  of 
an  unknown  input,  the  value  of  the  distribution  is  used  to  weight 
the  amount  of  change: 


j''Fold  +  fW*x 


M^new  “ 

j+f(x) 

(9) 

j=j-J-f(x) 

(10) 

A  similar  rule  is  used  for  the  variance.  The  variable  j  is  now  a 
floating  point  number  rather  than  an  integer,  but  the  two 
approaches  are  very  similar  in  their  implementations.  This  is 
essentially  the  Bayesian  approach  described  in  [7]. 

The  unsupervised  learning  rule  allows  the  system  to  continue 
to  learn  when  truth  data  is  unavailable,  producing  clusters  of 
"like"  reports,  A  typical  use  of  this  rule  would  be  to  first  teach 
the  system  on  known  inputs  using  the  supervised  learning  rule, 
then  allow  the  system  to  monitor  data,  modifying  its 
distributions  using  the  unsupervised  learning  rule. 


AxyG+1)  =  (xQ+i)  -  ^x(i))  *  (yG+1)  -  I^yG))  (5) 

Then 

1 

2^xy0+l)  =  -  SxyO'+l)  (6) 

j 

where  we  are  calculating  £xyG+^)>  the  x,y  component  of  the 

covariance  matrix  Z  after  the  j+l^t  input  clustered  in  this 
Gaussian.  xG)  and  yG)  are  the  x  and  y  component  of  the  input 
vector  after  the  jth  input,  and  |ix,  |iy  are  components  of  the 
mean.  These  formulas  require  the  node  to  store  j,  the  number  of 
points  in  the  node.  This  value  provides  the  weight  used  in  the 
sum  of  nodes  computed  for  the  final  layer:  j  divided  by  the  total 
number  of  points  in  the  class.  In  this  manner,  the  nodes 
corresponding  to  clusters  near  the  mean  of  the  distribution  are 
weighted  higher  than  nodes  corresponding  to  outliers. 

This  approach  can  also  be  made  to  choose  the  number  of 
nodes  in  the  hidden  layer.  When  a  point  comes  in,  the  hidden 
nodes  all  evaluate  their  response  to  the  input.  If  none  of  the 
nodes  fire  above  a  threshold,  it  is  assumed  that  none  of  the 
existing  nodes  properly  categorizes  the  input  and  a  new  node  is 
created.  Otherwise,  all  nodes  of  the  appropriate  class  that  fired 
above  a  threshold  are  allowed  to  adjust  their  parameters. 

One  of  the  advantages  of  this  system  is  that  it  is  dynamic.  If 
the  distribution  of  the  data  is  drifting  with  time,  the  network's 
distribution  will  also  drift. 


RESULTS 

The  kernel  estimator  was  compared  against  CARI  (Computer 
Assisted  Radar  Identification)  [3],  a  classification  progr^  based 
on  standard  statistical  techniques,  on  a  data  set  consisting  of 
4019  points  taken  from  63  hulls.  The  parameters  reported  are 
PRI  and  Scan.  The  two  systems  were  cross -validated  using  this 
data  set:  each  system  is  taught  on  4018  reports  and  tested  on  the 
missing  report.  If  the  hull  with  the  largest  value  on  the  output 
corresponds  to  the  hull  from  the  missing  report,  the  system  is 
said  to  be  correct.  If  the  second  highest  output  corresponds  to 
the  correct  hull,  the  system  gets  a  second  place,  and  if  the  third 
highest  is  correct  the  system  gets  a  third  place.  This  is  repeated 
until  the  system  has  been  tested  on  all  the  reports  in  the  data  set. 

The  results  are  tabulated  below.  Note  that  the  kernel 
estimator  is  slightly  better  on  this  data  set.  Preliminary 
indications  are  that  the  adaptive  mean  and  covariance  approach 
can  in  some  cases  improve  on  the  kernel  estimator,  but  in 
general  it  requires  more  points  to  construct  its  estimate. 


Method  Correct 
CARI  79% 
Kernel  82% 


Second  Third 
16%  3% 

12%  3% 


These  implementations  were  run  on  a  PC  AT  compatible 
computer.  It  should  be  noted  that  the  backpropagation  algorithm 
described  in  [4]  could  not  be  cross-validated  on  this  data  set, 
even  using  a  Cray,  due  to  the  prohibitive  processing 
requirements  of  the  learning  algorithm.  Although  this  is 
irrelevant  for  the  finished  system  (after  teaching),  it  is  an 
important  consideration  for  a  system  intended  to  ht  dynamic, 
and  hence  continually  learning. 


The  kernel  method,  and  the  moving  parameters  method,  can 
be  modified  to  allow  the  network  to  continue  in  an  unsupervised 
manner.  This  allows  the  system  to  continue  to  learn,  even 
though  truth  data  may  not  be  available.  This  is  important  in 
many  Navy  applications,  as  truth  data  is  often  hard  to  come  by, 
and  a  system  that  clusters  like  entities  may  be  the  best  that  one 
can  hope  for. 

The  idea  behind  the  unsupervised  learning  rule  is  that  if  the 
system  has  no  a  priori  knowledge  pertaining  to  the  class  of  the 
input,  it  should  update  each  class  proportionaly  to  the  likelihood 
that  the  input  comes  from  that  class.  This  is  indicated  in 
equations  7-10.  Equations  7  and  8  are  the  usual  update  rule  for 
the  mean  (note  that  this  is  written  in  a  less  efficient  but 
conceptually  simpler  form  than  equation  3): 


CONCLUSIONS 

The  classification  scheme  described  herein  developes  a 
statistical  model  of  the  input  entering  the  system.  In  adchtion, 
any  a  priori  knowledge  about  the  distribution  of  the  incoming 
data  can  be  incorporated  into  the  system.  This  yields  a 
powerful,  dynamic  classification  tool  possessing  the  advantages 
inherent  in  network  methodologies  and  yet  is  readily  understood 
via  statistical  analysis.  The  system  can  operate  in  both  a 
supervised  and  unsupervised  mode,  and  can  handle  missing  as 
well  as  noisy  data. 

The  gaussian  nodes  which  are  integral  to  the  system  are 
independent  of  one  another.  This  allows  each  node  to 
independently  perfomi  its  processing  based  on  its  inputs  and  its 
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activation  function.  In  keeping  with  traditional  neural  network 
schemes,  this  yields  an  easily  parallelizable  system.  Each  node 
can  be  thought  of  as  an  individual,  independent,  primitive 
processor.  The  nodes  in  various  layers  of  the  net  are  then  linked 
through  weighted  connections  to  nodes  in  the  previous  layer. 
Thus  the  processing  time  in  a  hardware  implementation  is 
independent  of  the  number  of  nodes  required  to  estimate  the 
density,  and  it  is  independent  of  the  number  of  hulls  in  the 
system. 

Another  aspect  of  the  system's  dynamic  nature  is  the 
adaptive  network  size.  Within  hardware  constraints,  gaussian 
nodes  can  be  allocated  dynamically,  as  dictated  by  the  statistics 
of  the  incoming  data.  Under  certain  learning  conditions,  many 
gaussian  nodes  may  be  necessary  to  capture  the  salient  features 
of  the  data.  At  other  times,  the  actual  distributions  being 
modeled  may  resemble  unimodal  gaussian  functions,  thereby 
requiring  very  few  gaussian  nodes. 

It  may  be  useful  to  consider  a  more  mature  system  in  which 
information  entering  the  system  at  different  times  from  multiple 
reports  can  be  used  to  strengthen  the  classification  process.  A 
temporal  classification  system  employing  the  gaussian  scheme 
described  herein  has  been  developed  [6].  While  one  report  may 
not  give  enough  information  for  a  classification,  a  track  of 
several  reports  may  give  more  information.  Another  application 
of  this  approach  would  be  to  look  at  the  signal  itself  rather  than 
measured  parameters  of  the  signal,  and  classify  it  based  on  the 
temporal  nature  of  the  signal. 
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Current  approaches  to  building  large  data  fusion  software  systems  involve  a  long  and  costly 
process:  years  may  elapse  between  the  initial  system  concept  to  the  time  that  the  system  is 
put  into  the  user’s  hands.  By  the  time  the  system  is  fielded,  the  technology  is  dated  and 
changes  or  improvements  to  the  system  may  be  years  in  coming.  Another  approach  to 
understanding  and  building  data  fusion  software  is  to  rapidly  build  prototypes,  obtain  user 
reaction  to  the  prototype,  and  iteratively  improve  the  system.  This  paper  describes  a  set 
of  rapid  prototyping  tools  we  have  developed  called  the  Knowledge  workbench.  The  Knowledge 
Workbench  is  a  flexible  testbed  for  developing  and  experimenting  with  data  fusion 
architectures;  it  includes  an  object-oriented  computing  environment,  a  knowledge-based 
expert  system  shell,  and  a  blackboard-style  architecture.  Collectively,  the  testbed  tools 
provide  the  capability  to  rapidly  prototype  a  data  fusion  system  by  giving  the  user  the 
ability  to  design  and  implement  a  system  architecture  using  knowledge  sources  to  capture  the 
functional  knowledge  and  message-passing  to  implement  the  system  data  flow.  A  knowledge 
source  contains  procedural  or  rule— based  knowledge  pertaining  to  a  function  within  the 
system;  it  is  encapsulated  as  an  object  in  the  computing  environment,  thereby  maintaining 
the  function’s  integrity  by  keeping  the  knowledge  in  a  modular  form.  We  illustrate  the  use 
of  the  Knowledge  workbench  by  describing  a  tactical  intelligence  analysis  system  prototype  we 
built  which  processes  blocks  of  TACREP  messages,  performs  analysis  on  the  contents  using  a 
set  of  knowledge  sources,  and  generates  the  intelligence  product  as  a  set  of  JINTACCS 
messages. 
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imoniicxxoN 

This  paper  describes  a  set  of  rapid 
prototyping  tools  we  have  developed  called 
the  Knowledge  Workbench.  The  Knowledge 
Workbench  is  a  flexible  testbed  for 
developing  and  experimenting  with  data  fusion 
architectures;  it  includes  an  object-oriented 
computing  environment,  a  knowledge-based 
expert  system  shell,  and  a  blackboard-style 
architecture.  Collectively,  the  testbed 
tools  provide  the  capability  to  rapidly 
prototype  an  intelligence  data  fusion  system 
by  giving  the  user  the  ability  to  flexibly 
define  "knowledge  components"  and  the  data 
flow  among  them.  A  prototype's  functional 
knowledge  is  encapsulated  in  a  set  of  loosely 
coupled  knowledge  components,  and  the  data 
flow  among  these  components  is  implemented 
using  the  message  passing  facilities  provided 
by  the  object-oriented  environment. 

The  underlying  goal  of  the  Knowledge 
Workbench  was  to  develop  a  flexible  set  of 
tools  for  use  in  prototype  development.  This 
goal  was  motivated  by  the  fact  that  most 
intelligence  data  fusion  systems  take  years 
to  develop,  and  once  developed  they  are  often 
already  severely  dated  both  in  terms  of 
technology  and  technique.  By  providing  a 
better  set  of  tools  for  prototype  development 
we  hope  to  improve  the  ability  to  experiment 
with  intelligence  data  fusion  architectures, 
thereby  improving  the  quality  of  the 
requirements  definition  for  such  a  system. 

We  will  first  present  the  motivation  for 
our  approach  to  developing  the  Knowledge 
Workbench  by  characterizing  the  data  fusion 
problem  and  how  it  is  viewed  from  the 
perspective  of  Distributed  Artificial 
Intelligence  (DAI) .  Next,  we  will  describe 
the  Knowledge  Workbench  architecture  by 
giving  an  account  of  the  system's  most 
salient  features  and  how  they  contributed  to 
the  testbed.  Finally,  we  will  demonstrate 
the  utility  of  the  Knowledge  workbench  by 
describing  a  prototype  that  was  built  using 
the  tools  in  the  testbed. 


There  were  tv/o  underlying  motivations 
for  the  design  of  the  Knowledge  Workbench; 

(1)  develop  a  set  of  tools  fitting  the 
characteristics  of  the  intelligence  data 
fusion  problem,  and  (2)  create  a  flexible 
testbed  environment  where  a  variety  of 
analytical  and  computational  techniques  can 
be  used  within  the  same  prototype.  In  this 
section  we  first  describe  how  we  characterize 
the  intelligence  data  fusion  problem  in  order 
to  explain  the  design  choices  we  made.  Then 
we  describe  some  technology  background 
which  relates  to  the  class  of  problems 
characterized  by  intelligence  data  fusion. 

lnt.ell.i.q^ce_j)ai;.a__Jg.usi^^ 

Characterization 

Our  view  of  the  intelligence  data  fusion 
process  derives  itself  from  the  functional 
and  organizational  perspective  of  Army 
tactical  intelligence  collection  and  analysis 
activities,  in  particular,  we  are  concerned 
with  systems  v/hich  can  be  used  to  support 
tactical  operations  V7hich  must; 

(a)  produce  a  view  of  the  battlefield 
using  single  discipline  intelligence 
analysis, 

(b)  combine  the  views  from  all  single 
discipline  intelligence  analysis 
activities  into  a  coherent  viev?  of  the 
battlefield,  and 

(c)  plan  the  use  of  sensing  resources  so 
as  to  maximize  their  effectiveness  in 
collecting  data  to  support  the  focus 
of  the  analysis. 

Each  of  the  analysis  activities  involves 
processing  streams  of  data  from  a  distributed 
set  of  sensors  or  sources  which  may  vary  in 
capability  and  type  and  which  provide  a 
temporally  disparate  view  of  the  battlefield. 
Taking  this  into  account,  "classical" 
intelligence  data  fusion  proceeds  through  a 
number  of  steps  whereby  the  sensor  data  is 
aligned,  associated,  correlated  and  fused 
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into  a  composite  object  or  view.  There  are 
a  number  of  different  approaches  to  the 
actual  sequence  of  steps  and  techniques 
applied  but  it  isn’t  our  intent  to  discuss 
them  here,  rather,  we  wish  to  point  out  the 
characteristics  of  the  intelligence  data 
fusion  problem  domain  which  led  to  our  design 
choices: 

(a)  Large  grain  problem  -  Although  this 
is  a  qualitative  description  of  the 
domain,  it  implies  that  the  fusion 
problem  is  large  enough  to  be  easily 
decomposed  into  a  set  of  sub¬ 
problems,  each  requiring  a  high 
degree  of  special  knowledge  or 
resources.  This  implies  that  the 
sub-problems  are  decomposed  along 
analytical  lines  so  that  a  high 
level  of  communication  with  other 
sub-problems  is  not  required.  Some 
readily  identifiable  intelligence 
data  fusion  sub-problems  are: 
correlation,  entity  identification, 
entity  attribute  analysis,  entity 
relationship  analysis,  aggregation, 
and  node  analysis. 

(b)  Loosely  Coupled  -  Once  decomposed, 
the  sub-problems  display  a  moderate 
to  high  degree  of  independence. 

There  are  some  interdependencies  in 
terms  of  the  sequence  of  processing, 
but  the  sub-problems  can,  in  some 
cases,  be  solved  concurrently, 
thereby  removing  the  need  to 
synchronize  the  sub-problems  into  a 
lock-step  sequence.  Instead,  the 
sub-problem  solvers  may  be  able  to 
opportunistically  update  or  query 
one  another  for  information, 

(c)  Data-driven  -  By  nature,  the  fusion 
problem  is  data-driven  due  to  the 
fact  that  there  is  a  constant  stream 
of  new  sensor  data  channeled  into 
the  fusion  center.  This  implies 
that  solutions  are  temporal  and 
open-ended,  and  a  solution  may  be 
triggered  or  changed  by  the  presence 
of  new  information. 

(d)  Variety  of  Processing  Paradigms  - 
Although  this  is  not  characteristic 
of  the  problem  domain,  rather,  of 
the  existing  approaches  to  dealing 
with  the  domain,  it  is  a  major 
consideration  for  building  a 
testbed.  The  current  approaches  to 
intelligence  data  fusion  include 
algorithmic  and  numeric  computation, 
fuzzy  or  probabilistic  reasoning, 
template  matching  and  knowledge- 
based  methods,  to  name  a  few.  No 
single  approach  will  solve  all  of 
the  sub-problems,  so  there  is  a  need 
to  construct  hybrid  systems  which 
take  advantage  of  the  applicable 
techniques  in  each  area. 

Distributed  Artificial  Intelliasnsg 

The  intelligence  data  fusion  problem 
characterization  fits  well  with  some  of  the 
approaches  to  problem  solving  used  in  the 


burgeoning  field  of  Distributed  Artificial 
Intelligence  (DAI) .  Much  of  the  research  in 
Artificial  Intelligence  (AI)  has  focused  on 
building  a  single  intelligent  agent  which 
solves  problems  using  heuristics,  knowledge- 
based  reasoning,  natural  language 
understanding,  planning,  or  theorem-proying. 
The  single  agent  paradigm  can  be  effective  in 
solving  problems  in  a  closed  domain,  but  it 
does  not  fare  as  well  in  cases  where  there 
are  multiple  interlinking  problems  in  a 
domain  spanning  several  areas  of  expertise, 
or  in  this  case,  intelligence  disciplines  and 
analytical  techniques.  For  example,  it  would 
be  unthinkable  to  implement  a  single  expert 
system  to  perform  all  of  the  intelligence 
data  fusion  functions  within  the  problem 
domain  we  mentioned  earlier. 

With  the  advent  of  parallel  computer 
architectures  and  advanced  networking 
techniques,  it  has  become  possible  to  create 
distributed  multi-agent  problem  solving 
systems.  Multi-agent  problem  solving  systems 
attempt  to  combine  information  from  a  diverse 
set  of  sources  to  solve  problems;  the  work  is 
divided  among  a  number  of  problem  solving 
components  which  cooperate  at  the  level  of 
dividing  and  sharing  knowledge  about  the 
problem  and  the  solution  state  (Ref,  1,  7). 

Some  of  the  early  work  in  DAI  resulted 
in  a  problem  solving  architecture  known  as  a 
Blackboard  System.  A  Blackboard  Architecture 
represents  partitions  of  expertise  in  modules 
called  knowledge  sources.  The  knowledge 
sources  attempt  to  cooperatively  solve  a 
global  problem  by  sharing  intermediate 
problem  solutions  on  a  central  data  structure 
known  as  a  blackboard.  They  communicate  with 
one  another  indirectly  in  that  all  of  their 
communication  is  done  via  the  blackboard 
structure.  A  knowledge  source  monitors  the 
blackboard  with  a  set  of  pattern  triggers; 
if  there  is  a  match  between  a  knowledge 
source’s  pattern  trigger  and  some  data  on  the 
blackboard  then  a  control  mechanism  is 
notified.  After  all  of  the  knowledge  sources 
have  an  opportunity  to  view  the  blackboard, 
the  control  mechanism  chooses  which  knowledge 
source  to  activate  based  on  some  conflict 
resolution  criteria.  Blackboards  have  been 
used  to  implement  a  number  of  different 
systems  including  the  Hearsay  II  project 
(Ref. 4)  and  the  Distributed  Vehicle 
Monitoring  Testbed  (Ref.  5) . 

In  conclusion,  the  goals  and  objectives 
of  DAI  are  very  closely  aligned  with  the 
characteristics  of  the  intelligence  data 
fusion  problem  domain.  The  DAI  concept  of  a 
knowledge  source  can  be  used  to  implement  the 
sub-problems  which  resulted  from  decomposing 
the  large  grain  fusion  problem.  Since  the 
fusion  sub-problems  tend  to  be  loosely 
coupled,  the  DAI  concept  of  blackboard 
problem  solving  seems  appropriate  also.  In 
fact,  we  have  adopted  some  of  the  concepts 
from  the  blackboard  approach  to  problem 
solving  because  of  the  leverage  it  provides 
in  terms  of  partitioning  the  large  grain  data 
fusion  problem  into  a  set  of  what  we  call 
knowledge  components  and  the  concept  it 
provides  of  cooperative  problem  solving.  Two 
of  the  primary  differences  between  a 
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classical  blackboard  architecture  and  our 
approach  are  (1)  the  way  in  which  the 
knowledge  components  communicate,  and  (2)  how 
they  are  activated.  This  is  discussed  in 
more  detail  later  in  the  paper. 


mQMLEDGE..WQRKBENCH  ARCHITECTDRE 

As  previously  stated,  the  underlying 
goal  of  the  Knowledge  Workbench  is  to  provide 
a  flexible  testbed  for  constructing,  testing, 
and  modifying  intelligence  data  fusion 
prototypes.  In  order  to  meet  this  goal  we 
had  to  develop  the  testbed  architecture  so  as 
to  account  for  the  characteristics  of  the 
data  fusion  problem  domain,  while  at  the  same 
time  exploiting  current  approaches  to  problem 
solving  and  computation  as  it  has  been 
developed  in  DAI  and  other  disciplines. 

Using  the  metaphor  of  a  workbench,  we 
see  the  testbed  as  a  way  of  constructing  an 
intelligence  data  fusion  prototype  in  the 
same  way  as  one  might  assemble  a  piece  of 
furniture;  first  the  parts  are  made  and  then 
they  are  fastened  together.  The  added 
requirement  for  the  testbed  is  that  one 
should  be  able  to  disassemble  the  prototype 
and  connect  the  pieces  another  way.  in  the 
description  that  follows  we  will  explain  how 
we  model  the  fusion  sub-problems  as 
"knowledge  components"  in  the  architecture; 
the  knowledge  components  are  "glued  together" 
by  the  data  flow.  The  ability  to  treat  the 
knowledge  components  as  self-contained  and 
modular  is  provided  by  the  object-oriented 
environment;  in  addition,  the  data  flow  is 
implemented  using  the  message  passing 
facilities  also  provided  by  the  object- 
oriented  environment.  The  result  is  that  the 
Knowledge  Workbench  provides  a  flexible 
testbed  for  prototype  development,  we  define 
flexible  as  (1)  the  ability  to  insert, 
remove,  or  exchange  knowledge  components 
within  a  prototype  architecture,  and  (2)  the 
ability  to  easily  change  the  data  flow  among 
knowledge  components. 

Qfclect-Qfiented  Env_ironment 

One  of  the  critical  design  choices  we 
made  for  the  Knowledge  Workbench  was  to  embed 
the  testbed  in  an  object-oriented 
environment.  The  reason  this  choice  was 
critical  is  that  the  object-oriented 
programming  paradigm  underlies  the  entire 
system  and  provides  it  with  many  of  the 
features  required  for  the  intelligence  data 
fusion  problem  characterized  above. 

Object-oriented  programming  is  a  concept 
that  was  first  realized  in  SIMULA,  a 
simulation  language,  and  has  since  been  used 
in  other  languages  such  as  Smalltalk,  Flavors 
and  Loops.  Smalltalk  goes  to  an  extreme  in 
terms  of  providing  a  totally  object-oriented 
environment  (Ref.  3) ,  while  Flavors  and  Loops 
are  object-oriented  features  developed  for 
use  in  the  Lisp  environment.  Our  object- 
oriented  shell  was  developed  to  simulate  the 
Time  Warp  distributed  operating  system  (Ref. 


The  basic  component  of  an  object-oriented 
system  is  the  object.  An  object  is  an  entity 
which  combines  the  properties  of  data  and 
procedure:  it  preserves  local  state  and  it 
performs  computation  (Ref,  8).  In  keeping 
with  this  definition,  all  of  the  problem 
solvers  and  data  in  the  Knowledge  Workbench 
are  implemented  as  objects.  All 
communication  among  objects  is  through  the 
message  passing  facilities  provided  by  the 
object-oriented  shell.  Hence,  all  activity 
in  the  Knowledge  Workbench  is  initiated  by 
passing  a  message  to  a  knowledge  component  as 
opposed  to  invoking  a  procedure. 

Memory  is  not  shared  among  objects;  each 
object  has  a  private  data  space.  In  order  to 
share  data,  an  object  must  send  the  data  in  a 
message  to  another  object.  In  this  way, 
objects  cannot  directly  access  the  data  space 
of  one  another.  The  messages  are  time- 
stamped  with  the  sending  and  receiving  times, 
and  they  also  contain  the  names  of  the  sender 
and  receiver.  The  time  stamps  are  used  to 
synchronize  the  activation  of  the  objects; 
an  object  can  only  read  a  message  when  the 
global  time  equals  the  receiving  time  of  the 
message. 

Knowledge  Components 

Data  fusion  functions  are  conceptually 
treated  as  "knowledge  components"  on  the 
Workbench,  A  data  fusion  "knowledge 
component"  can  be  either  procedural  or 
declarative  in  nature,  thereby  allowing  the 
prototype  engineer  to  mix  algorithmic  and 
probabilistic  techniques  with  knowledge-based 
approaches  to  problem  solving  within  the 
same  prototype.  In  a  practical  sense,  a 
knowledge  component  is  implemented  as  an 
object  within  the  testbed  environment,  thus, 
it  must  be  designed  in  such  a  way  that  it 
will  use  message  passing  as  its  means  of 
communication  with  other  knowledge 
components,  in  addition,  the  knowledge 
component  will  have  a  private  set  of  data 
which  can  be  stored  as  a  part  of  its  state 
when  it  is  not  active.  The  net  effect  of 
partitioning  the  intelligence  data  fusion 
problem  solvers  into  knowledge  components  in 
the  object-oriented  environment  is  that  the 
knowledge  components  can  serve  as  the  basic 
building  blocks  for  building  a  data  fusion 
architecture.  Thus,  a  prototype  engineer  can 
insert,  delete,  or  interchange  the  knowledge 
components  in  a  modular  manner;  this 
capability  is  magnified  further  by  the  fact 
that  the  data  flow  among  knowledge  components 
is  easily  manipulated  due  to  the  message 
passing  approach  to  communication. 

The  knowledge  components  work  on 
different  parts  of  the  same  global  problem 
(e.g.  situation  assessment  of  the 
battlefield) .  Thus,  the  goal  is  to  develop 
the  knowledge  components  is  such  a  way  that 
their  actions  are  interrelated  and 
cooperative  in  nature.  Communication  among 
knowledge  components  is  handled  through 
message  passing  as  provided  by  the  object 
oriented  environment.  The  messages  are  time- 
stamped,  thus  the  synchronization  of  actions 
among  knowledge  components  does  not  have  to 
be  programmed  by  the  application  developer. 
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rather,  it  is  handled  by  the  object-oriented 
shell. 

In  addition  to  communicating  directly 
with  one  another  through  message  passing,  the 
knowledge  components  may  also  communicate 
indirectly  by  sending  messages  to  a 
blackboard  defined  by  the  prototype 
developer.  Hence,  the  Knowledge  Workbench 
provides  two  means  of  communication  for  the 
knowledge  components:  (1)  direct  message 
passing,  and  (2)  the  blackboard. 

Inference  Enciina 

The  testbed  also  includes  an  inference 
engine  which  can  be  used  to  process  the  rule- 
based  knowledge  components.  The  typical 
expert  system  has  three  components:  a  rule 
base,  a  fact  base  and  an  inference  engine. 
Expert  systems  shells  usually  provide, the 
entire  context  from  which  a  system  is  run, 
with  non-rule  based  procedures  being  callable 
by  the  expert  system,  but  not  invocable  apart 
from  it. 

The  approach  in  the  Knowledge  Workbench 
is  to  make  the  expert  system  tools  available 
without  subsuming  the  entire  computational 
context.  In  this  way  the  rule-based 
components  blend  in  with  the  other 
programming  paradigms  in  the  system.  Because 
of  this  philosophy  of  making  the  inference 
engine  a  resource  instead  of  a  context 
controlling  driver,  the  rule  bases  are  stored 
in  knowledge  components  and  fact  bases  are 
loaded  onto  a  blackboard.  Both  of  these 
items  are  implemented  as  objects  from  the 
system *s  point  of  view,  hence  the 
communication  by  message  passing  is  enforced, 
thereby  allowing  us  to  break  apart  the  expert 
system  into  component  parts.  This  permits  us 
to  create  a  mixed  computing  paradigm  to 
combine  the  aspects  of  rule-based  computing 
with  numeric  and  algorithmic  processing. 

The  inference  engine  was  designed  to 
operate  primarily  in  a  forward  chaining  mode. 
This  choice  of  inference  technique  fits  the 
data-driven  nature  of  a  fusion  system.  A 
forward  chaining  inference  engine  begins  with 
a  set  of  facts  or  premises  and  seeks  to  make 
inferences  or  conclusion  based  on  these 
facts.  This  is  in  contrast  to  a  backward 
chaining  system  where  the  engine  is  given  a 
goal  or  proposition  and  must  prove  that  it  is 
true  based  on  the  facts  and  rules  which  are 
in  the  knowledge  base.  The  forward  chaining 
approach  to  rule  based  processing  provides  a 
means  of  making  all  possible  inferences  about 
a  set  of  sensor  or  correlated  data  thereby 
finding  all  of  the  identifiable  battlefield 
entities,  not  just  selected  ones.  This  does 
not  mean  that  it  is  not  possible  to  set  a 
higher  priority  on  the  search  for  particular 
items,  for  this  can  be  done  by  partitioning 
the  rules  into  categories  by  priority,  type, 
or  any  other  criteria.  In  fact,  one  of  the 
features  of  the  inference  engine  is  the 
ability  to  partition  the  rules  in  a  given 
knowledge  component  into  sets  which  can  be 
loaded  or  unloaded  depending  on  the  context. 
The  context  is  monitored  by  meta-level  rules 
defined  by  the  user,  and  when  a  particular 
condition  is  detected  the  set  of  active  rules 


may  be  totally  changed.  Using  the  same 
approach  at  a  knowledge  component  level,  a 
message  may  be  sent  from  one  knowledge 
component  to  another  as  a  result  of  a  rule 
invocation.  The  message  may  contain  data  or 
else  it  may  contain  an  activation  message. 

The  inference  engine  handles  the 
following  rule  features  when  checking  the 
precondition  of  a  rule: 

(a)  AND  “  conjunctions  of 

clauses 

(b)  OR  -  disjunctions  of 

clauses 

(c)  NOT  -  negation  of  a  clause 

(d)  *expr*  -  a  predicate 

expression  to  be  evaluated 

and  treated  as  true 

or  false. 

Once  a  ruled  precondition  is  satisfied,  the 
rule  can  be  "fired"  and  some  action  can  be 
taken.  These  actions  include: 

(a)  ADD  -  add  a  proposition 

clause  to  the  blackboard 

(b)  DELETE  -  remove  a 

proposition  from  the 

blackboard 

(c)  ASK  -  ask  the  uses  for 
information 

(d)  @  -  make  a  procedure 

call  or  send  a  message  to 

another  knowledge 

component 

(e)  LOAD  -  change  the  rule- 

base  by  loading  new 

partition (s) 

The  individual  clauses  in  a  rule  may 
contain  pattern  variables.  The  inference 
engine  handles  all  of  the  variable  bindings 
and  instantiation  while  doing  the  pattern 
matching  in  the  system.  In  addition, 
pattern  matching  functions  are  made  more 
efficient  through  the  use  of  a  set  of  hash 
table  retrieval  functions.  This  is  described 
in  more  detail  in  the  next  subsection. 

Blackboard  Object 

A  blackboard  in  the  Knowledge  workbench 
is  a  data  object.  It  receives  input  data  in 
two  different  ways: 

(1)  the  user  can  load  a  priori  data  from  a 
file  onto  the  blackboard  using  a  menu 
option  on  the  Knowledge  Workbench, 

(2)  the  knowledge  components  may  send  data 
to  the  blackboard  -  this  data  from  the 
knowledge  components  may  be  correlated 
or  preprocessed  data,  or  it  may  be  in 
the  form  of  newly  inferred 
information. 

The  blackboard  is  implemented  using  a  nash 
table  to  make  the  storage  and  retrieval  of 
information  more  efficient.  The  hash 
function  operates  on  the  second  element  of 
each  clause,  which  is  normally  some  sort  of 
relational  descriptor.  For  example,  in  the 
clause: 

{?nodel  controls  ?netl) , 
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(a)  COUNT  {ENTITIES  1  ENTITY-REUTIONSHIP-GRAPHS  I  NODES) 

&  OPTIONAL  filter 

(b)  ENTITY-FILTER  &  OPTIONAL  reset 

(c)  ENTITIES  (entity-id  I  ENTiTIES  I  ALL-ENTITIES) 

(d)  EXPLAIN  function-name 

(e)  LIST-FUNCTIONS 

^  MEMBERS-OF  (entity-relatlonship-ciraph 

I  ENTITY-RELATIONSHIP-GRAPHS 
I  ALL  -GRAPHS  I  node-name  I  NODES 
I  ALL-NODES) 

&  OPTIONAL  {ENTITIES  !  ALL-ENTITIES) 

(g)  MEMBERSHIP-OF  {entity-id  I  ENTiTIES  I  ALL  ENTITIES) 

(h)  GRAPH-FILTER  &  OPTIONAL  reset 

(i)  GRAPH-MENBEflSHIP-OF  {entity-id  I  ENTITIES) 

I  ALL-ENTITIES) 

(j)  GRAPH  {graph-name  I  GRAPHS  I  ALL-GRAPHS) 

(k)  GRAPH-RELATED-TO  {node-name  1  NODES  I  ALL-NODES) 

(!)  NODE-FILTER  &  OPTIONAL  reset 

(m)  NODE-MEMBERSHIP-OF  {entity-id  I  ENTITIES 

I  Aa-ENTITIES) 

(n)  NODES  {node-name  I  NODES  I  ALL-NODES) 

(O)  NOD ES-RELATED-TO  {graph-name  I  GRAPHS  I  ALL-GRAPHS) 

(P)  QUfT  ^ 

(q)  SHOW-FILTERS 


Figure  1.  Order  of  Battle  Database  Query  Language 


KNOWLEDGE  WORKBENCH 


SYSTEM  OPERATION 
1-  PROCESS  NE)a  MESSAGE  BLOCK 
NEXT  BLOCK:  [090900) 

LAST  BLOCK:  (091230) 

2  -  RUN  EXPERT  SYSTEM  SHELL 


ENGINEEflING/SYSTEM  PARAMETERS 

1 0  -  SET  MESSAGE  BLOCK  TIME  SIZE  {30  min) 

11  -  ENTER  FEBA  TRACE  :  [AS  OF  090900) 

12  -  EDIT  RULE  BASE 

13  -  LOAD  FILE 

14  -  ACTIVATE  RULE  BASE:  TWNAKB’ 

1 5  -  ACTIVATE  BLOCKBOARD:  'AAA* 

16  -  LOAD  DATA  ONTO  BUCKBOARD:  LOADED 

17  -  RESET  RULES/BUCKBOARD 

ENTER  CHOICE: 


DATA  ACCESS 

20  -  QUERY  ORDER  OF  BATTLE  DB 

21  -  QUERY  BLACKBOARD 

22  -  DISPUY  INPUT  MESSAGES 

23  -  DISPUY  OUTPUT  MESSAGES 
24 -DISPUY  FEBA  TRACE 

25  -  DISPUY  EXPERT  SYSTEM  STATS 

OUTPUT  SWITCHES 

30  -  APPLICATION  MESSAGES:  OFF 

31  -  OBJECT  ACTIVATION:  OFF 
32 -DEBUG  MODE: OFF 

33  -  SESSION  OUTPUT  TO  FILE:  OFF 

34  -  DATA  ACCESS  OUTPUT  TO:  SCREEN 

q-Qurr 


Figure  2.  Knowledge  Workbench  User  Interface 


SENSOR  MESSAGES 


ANALYTICAL 

DISPUY 

SYSTEM 


FUSION 

PRODUCT 

MESSAGES 


Figures.  Fusion  Prototype 
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"?nodel"  and  "?netl"  are  pattern  variables 
and  "controls"  is  the  relational  descriptor 
for  the  two  variables.  It  should  also  be 
noted  that  only  instantantiated  clauses  are 
stored  on  the  blackboard,  hence,  the  two 
pattern  variables  "?nodei"  and  "?netl"  could 
be  replaced  by  actual  node  and  net 
identifiers.  When  there  are  multiple 
instances  of  clauses  with  the  same  relational 
descriptors,  a  list  is  formed;  the  inference 
engine  accesses  a  clause  list  from  the 
blackboard,  and  it  searches  the  list  for  a 
match  with  the  current  precondition  clause. 
Hence , 

(?entity  part-of  ?node) 

matches; 

(id-1000  part-of  node-10) 

where  ?entity  is  bound  to  id  1000,  and  ?node 
is  bound  to  node-10, 

Message  Ptopeasins 

Sensor  data  is  sent  to  a  data  fusion 
prototype  in  the  testbed  from  a  file  of 
formatted  sensor  messages.  In  the 
application  we  built  using  the  Knowledge 
Workbench,  the  file  of  formatted  sensor 
messages  was  broken  into  time  interval  blocks 
as  specified  by  the  user  with  the  menu 
selection  for  block  size.  Once  the  block 
size  is  specified,  the  messages  from  that 
block  can  be  read  into  the  application  and 
parsed  for  use  by  the  knowledge  components. 
Thus,  it  is  possible  to  handle  varying  sized 
blocks  of  messages  as  long  as  they  are  sent 
to  the  input  file. 

Output  message  processing  is  handled  in 
a  similar  way  to  input  message  processing. 
Once  there  is  sufficient  information  about  a 
battlefield  entity  or  node  in  the  order  of 
battle  database,  a  formatted  message  is 
generated  containing  information  about  the 
entity,  its  location,  and  its  activity.  The 
knowledge  component  which  formats  the  order 
of  battle  data  also  stores  the  message  to  a 
file. 

Order  of  Battle  Database ..OMect 

Another  special  type  of  data  object  in 
the  Knowledge  workbench  is  the  Order  of 
Battle  Database,  This  database  is 
initialized  with  a  priori  data  about  the 
opposing  force’s  strength  and  disposition 
profile;  it  is  subsequently  populated  by  the 
knowledge  components  responsible  for 
identifying  entities,  entity  attributes, 
entity  relationships,  and  nodes.  After  each 
cycle  in  which  a  block  of  data  messages  has 
been  processed,  the  user  can  make  queries  on 
the  Order  of  Battle  database  using  a  stylized 
query  language.  See  Figure  1  for  a  list  of 
the  queries  possible  in  the  system. 

Testbed  Environment  Switches 

The  Knowledge  Workbench  user  interface 
provides  a  number  of  switches  and  options  for 
running,  testing,  and  debugging  a  prototype. 


The  testbed  can  be  used  in  several  different 
modes.  As  a  prototype  developer,  the  user 
can  edit  a  rule-based  knowledge  component  and 
run  it  separately  from  a  full  prototype 
simply  by  selecting  the  menu  options  to  edit 
the  rule  base,  activate  a  rule-base,  activate 
a  blackboard,  load  data  onto  a  blackboard, 
and  run  the  expert  system  shell.  Hence,  it 
is  possible  to  use  the  Knowledge  Workbench  to 
perform  knowledge  engineering  activities  on  a 
rule-based  knowledge  component. 

When  debugging  a  prototype  there  are  a 
number  of  different  levels  of  output  from  the 
system  ranging  from  no  output  at  all  to  a 
step-by-step  processing  of  the  data.  When 
interested  in  the  knowledge  component 
application,  the  user  may  toggle  application 
messages  on  and  off.  When  the  focus  is  on 
monitoring  the  data  flow,  the  user  has  two 
options:  (1)  show  the  activation  of  the 
different  knowledge  component  objects,  or  (2) 
actually  step  through  the  prototype  cycle  a 
message  at  a  time.  In  the  step  mode  the  user 
can  view  the  message  contents  as  well  as  the 
objects  state. 

Finally,  the  testbed  provides  the  means 
to  view  the  various  database  objects  in  the 
prototype  as  well  as  the  input  and  output 
messages  and  the  expert  system  statistics. 


User  Interface 

The  interface  to  the  testbed  is  shown  in 
Figure  2 .  The  interface  was  developed  to 
allow  the  user  to  operate  the  testbed  in  two 
different  modes;  (1)  Knowledge  Engineering 
and  (2)  System  Operation,  In  the  knowledge 
engineering  mode  the  user  can  choose  to  edit 
the  rule  base,  load  data  onto  the  blackboard, 
and  test  the  rules  by  running  the  expert 
system  shell  separate  from  the  prototype. 
Prior  to  normal  system  operation,  the  user 
has  the  ability  to  choose  the  next  block  of 
formatted  messages  to  be  read  and  processed 
by  the  system.  Once  the  user  starts  the 
prototype  it  will  process  a  message  block 
until  all  knowledge  components  have  completed 
their  activations. 


AN  INTELLIGENCE  DATA  FUSION  PROTOTYPE 

An  example  of  an  actual  fusion  prototype 
architecture  and  its  data  flow  is  shown  in 
Figure  3,  This  prototype  was  implemented 
using  the  Knowledge  workbench  and  currently 
runs  in  Lisp  on  a  uniprocessor  system. 

Problem  DecomPOsiti^_n 

The  prototype  was  created  with  a  set  of 
knowledge  components  to  (1)  process  messages, 
(2)  perform  limited  self-correlation,  (3) 
construct  entity  relationship  graphs,  (4) 
perform  analysis  of  entity  attributes,  (5) 
perform  keyword  and  activity  analysis,  (6) 
and  aggregate  groups  of  entities  into  nodes. 
In  addition,  there  were  two  knowledge 
components  used  to  store  information,  namely, 
the  blackboard  and  the  Order  of  Battle 
database. 
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The  input  sensor  data  is  stored  as  a  set 
of  formatted  messages  in  a  file;  they  are 
generated  from  a  simulation  in  which  the 
opposing  forces  are  making  a  transition  from 
one  scenario  to  another.  The  user  runs  the 
prototype  from  the  testbed  interface  shown  in 
Figure  2.  The  messages  are  processed  in  time 
blocks  of  a  user-specified  size,  and  the  new 
information  is  correlated  with  pre-existing 
knowledge  about  the  entities  being  analyzed. 

Data  Flow 

The  data  flow  in  the  prototype  is  shown 
by  the  lines  connecting  the  knowledge 
components  in  Figure  3.  The  data  flow  can 
easily  be  modified  by  merely  changing  the 
addressee  in  the  message.  One  of  the 
problems  we  encountered  during  the 
implementation  of  this  prototype  was  keeping 
a  definition  of  how  each  knowledge  component 
expects  its  data  messages  to  be  formatted. 

We  recommend  that  a  message  format  dictionary 
be  maintained  for  each  knowledge  component  in 
order  to  minimize  the  problems  associated 
with  changing  the  data  flow.  In  addition, 
each  knowledge  component's  data  dependencies 
should  be  documented  since  it  may  never  be 
activated  if  it  doesn't  receive  the  data  it 
needs . 

Computational  Paradigms 

Included  in  the  prototype  were  knowledge 
components  which  used  (l)  'c*  functions  to  do 
message  parsing,  (2)  rule  based  systems  to 
recognize  entities,  nodes,  and  their 
relationships,  (3)  a  blackboard  for 
cooperative  problem  solving,  (4)  a  graph 
analysis  algorithm, 

(5)  a  pattern  analysis  algorithm,  and  (6)  an 
Order  of  Battle  database. 


FUTPRE  RESEARCH 

There  are  a  number  of  issues  we  wish  to 
address  in  our  next  iteration  of  the 
Knowledge  Workbench.  The  next  obvious  step  is 
to  extend  the  tool  to  operate  on  a  set  of 
distributed  nodes  instead  of  on  a 
uniprocessor.  The  current  version  of  the 
object-oriented  shell  simulates  the  Time  Warp 
distributed  operating  system,  hence,  the 
knowledge  sources  will  not  have  to  be 
modified;  the  primary  modification  will  be 
distribute  the  object-oriented  shell  over  a 
network  of  computers. 

In  addition  to  distributing  the 
^owledge  Workbench,  we  are  interested  in  the 
issue  of  creating  knowledge  components  that 
are  able  to  reason  about  one  another  in  order 
to  form  coordination  networks;  a 
coordination  network  may  be  formed  for  the 
purpose  of  sharing  knowledge,  goals,  skills, 
and  plans  to  jointly  take  an  action  or  solve 
a  problem  <Ref.  2).  Such  a  collection  of 
intelligent  agents  could  eliminate  the  need 
for  the  prototype  developer  to  modify  the 
intelligence  data  fusion  architecture  since 
it  would  modify  itself  according  to  the 
situation. 


The  Knowledge  Workbench  was  a  team 
effort.  The  object-oriented  shell  was 
developed  by  Steve  Hughes.  The  Order  of 
Battle  Database  object  and  query  langauge 
were  developed  by  Mike  Becker.  Jeff  wike  was 
responsible  for  implementing  the  sensor 
message  database,  the  scenario  for  the 
prototype,  and  several  knowledge  components. 
Others  who  made  contributions  are  Shig 
Hozaki,  Richard  Benninger,  and  will  Duquette, 

This  work  was  sponsored  by  the  United 
States  Army  Intelligence  Center  and  School 
(USAIC&S)  at  Ft,  Huachuca,  Arizona,  under 
contract  number  J28-82.  The  views  and 
opinions  presented  in  this  paper  are  entirely 
our  own  and  do  not  necessarily  represent  the 
views  of  USAIC&S  or  the  U.S.  Army. 


REFERENCES 

1.  Alan  Bond,  Les  Gasser,  "Readings  in 
Distributed  Artificial  Intelligence",  Morgan 
Kaufman  (1988) .  (forthcoming) 

2.  Les  Gasser,  Randall  w.  Hill,  Jr.,  Jon 
Lieb,  Nicolas  Rouquette,  "ICE  Part  1:  iCE  as 
a  General  Framework  and  Problem  for 
Distributed  AI",  USC  DAI  Group  Research  Note 
45,  Dept,  of  Computer  Science,  USC,  March, 
1988. 

3.  Adele  Goldberg,  David  Robson, 
"Smalltalk-80;  The  Language  and  Its 
Implementation",  Addison-Wesley ,  Reading, 
Massachusetts  (1987) , 

4.  Lee  D.  Erman,  Frederick  Hates-Roth, 
Victor  R,  Lesser,  D.  Raj  Reddy,  "The  Hearsay 
II  Speech-Understanding  System:  Integrating 
Knowledge  to  Resolve  Uncertainty",  Computing 
SUr.YOy^,  Vol.  12,  No.  2,  June  1980. 

5.  Victor  Lesser,  Daniel  D.  Cor kill, 

"The  Distributed  Vehicle  Monitoring  Testbed: 

A  Tool  for  Investigating  Problem  Solving 
Networks",  Ar  Magazine.  Fall  1983. 

6.  David  Jefferson,  H,  Sowrizral,  "Fast 
Concurrent  Processing  using  the  Time  Warp 
Mechanism",  SCG  Conferences  on  Distributed 
SJjnUlStifflD,  San  Diego,  January  1905. 

7.  Reid  G.  Smith,  "A  Framework  for 
Distributed  Problem  Solving",  UMI  Research 
Press,  Ann  Arbor,  Michigan,  1981, 

8.  Mark  Stefik,  Daniel  G.  Bobrow, 
"Object-Oriented  Programming;  Themes  and 
Variations",  AI  Magazine,  winter  1986,  pp. 


444 


Integrating  Plans  and  Scripts: 

An  Expert  System  for  Indications  and  Warning^ 


John  W.  Benoit 
Edward  J.  Dombroski 
Pamela  W.  Jordan 
Sharon  J.  Laskowski 

The  MITRE  Corporation 
C^I  Artificial  Intelligence  Center 
7525  Colshire  Drive 
McLean,  Virginia  22102 


An  expert  system  for  monitoring  complex  activity  in  order  to  predict  future  actions 
of  intelligent  agents  requires  knowledge  that  is  difficult  to  capture  in  standard  rule- 
based  representations.  This  paper  describes  the  artificial  intelligence  methodology  we 
have  developed  for  plan  recognition  that  leads  to  the  design  of  such  an  expert  system 
decision  aid.  The  application  domain  motivating  this  research  is  tactical  battle 
management  where  an  expert  system  can  aid  intelligence  analysts  in 

the  battlefield  more  quickly  and  accurately.  However,  this  research  can  be  applied  to 
any  system  that  is  monitoring  a  volatile  environment  with  incomplete  and/or  misleading 
information . 

To  predict  actions,  plausible  goals  and  plans  must  be  determined  based  on  observed 
events  and  the  order  in  which  the  events  have  occurred  while  taking  into  consideration 
that  incoming  information  is  often  sparse  or  deceptive.  Our  research 

in  script-like  format  and  artificial  intelligence  planning  techniques  to  formulate  a  set 
of  hypotheses  of  the  adversary's  actions.  We  have  focused  on  integrating  a  script-based 
goal  detector  with  a  plan  recognizer  to  generate  feasible  plans.  By  specifying  tactical 
and  doctrinal  sequences  of  events,  the  script  representation  enables  the  generation  of  a 
set  of  hypotheses  about  the  adversary’s  goals.  Observed  events  are  compared  to  stereotypical 
seguences  of  events  using  flexible  temporal  matching  techniques  that  we  have  developed. 

The  hierarchical  plan  recognizer  generates  plans  from  this  set  of  hypotheses 
mation  about  the  particular  situation.  By  limiting  the  number  of  hypotheses  through  script 
matching,  the  planning  becomes  computationally  tractable.  The  multiple  pla^s  are  generated 
and  organized  using  assumption-based  truth  maintenance  techniques  tailored  for  use  in  the 
generation  and  expansion  of  partial  plans. 

The  resulting  expert  system,  IPS,  can  suggest  several  possible  stereotypical  goals  as 
explanations  of  the  sequences  of  events  unfolding  and,  then,  detail  the  plan  steps  that  the 
adversary  must  execute  in  order  to  obtain  these  goals.  The  key  contributions  of  this 
research  are  the  combination  of  script  and  plan  knowledge  representations  in  the  context 
of  an  expert  system,  the  temporal  reasoning  applied  to  matching  observed  events  against 
stereotypical  scripts,  and  the  representation  of  plan  alternatives  in  a  hierarchical 
planner.  IPS  itself  suggests  ways  in  which  expert  system  technology  can  aid  the  human 
analyst  at  many  levels  of  problem  solving. 
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Introduction 

This  paper  describes  our  current  research  and  development 
of  artificial  intelligence  techniques  and  structures  required  to 
build  an  expert  system  decision  aid  that  can  predict  the  goals 
zind  2u:tions  of  various  intelligent  actors.  The  application  do¬ 
main  in  which  these  techniques  are  being  developed  is  tactical 
battle  management  where  such  an  expert  system  can  aid  in¬ 
telligence  analysts  in  determining  trends  on  the  battlefield. 
However,  this  research,  when  complete,  can  be  applied  to  any 
system  that  is  monitoring  a  volatile  environment  with  incom¬ 
plete  and/or  misleading  data. 

The  main  objective  of  this  project  is  to  demonstrate  that 
the  use  of  artificial  intelligence  techniques  based  on  scripts 
and  planning  will  greatly  assist  intelligence  analysts  in  the 
detection  and  identification  of  actions  evolving  over  time.  A 
method  for  the  early  detection  of  goals  is  to  monitor  a  num¬ 
ber  of  continuing  phenomena,  for  example,  troop  movements, 
and  signal  an  alert  when  any  of  these  phenomena  exhibit  val¬ 
ues  outside  a  preselected,  “normal”  range.  These  phenomena 
are  usually  called  indicators.  Based  on  an  indicator  alert,  an¬ 
alysts  must  hypothesize  possible  missions  from  a  review  of 
past  behavior,  doctrine,  and  previous  reports.  In  doing  so, 
the  analysts  attempt  to  fit  the  reports  to  plausible  plans  for 
achieving  the  missions.  Prom  the  candidate  hypotheses  and 
their  associated  plans,  it  is  possible  to  predict  events  and  the 
related  intelligence  Indications  associated  with  each  of  these 
hypotheses  which  may  be  compared  with  future  reports  in 
order  to  identify  the  most  likely  action. 

This  analysis  can  be  viewed  as  a  type  of  plan  recogni¬ 
tion.  Applicable  artificial  intelligence  techniques  have  been 
explored  in  other  fields  of  AI,  such  as  deep  semantic  analy¬ 
sis  of  natural  language  stories  and  automated  planning.  Key 
to  the  application  of  these  mechanisms  to  the  intelligence  do¬ 
main  is  the  representation  of  likely  events  of  possible  crises 
with  a  knowledge  structure  that  allows  efficient  matching  of 
actual  indicator  results  to  these  events.  For  small  domains,  it 
is  conceivable  to  generate  all  possible  plans  and  match  these 
against  the  observations.  However,  for  any  domain  of  even 
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slight  complexity,  this  leads  to  an  intractable  combinatorial 
explosion  of  plans  and  plan  variations.  This  is  especially  true 
in  our  case  where  the  variations  cire  extremely  large.  This  calls 
for  some  method  of  pruning  the  number  of  likely  missions  and 
plan  variations  before  enemy  plan  generation  is  begun.  The 
idea  of  a  script  representation  from  Schank’s  research  in  nat¬ 
ural  language  processing  is  applicable  here  [SA77].  A  script 
is  a  formalized,  stereotypical  representation  of  a  sequence  of 
actions  oriented  toward  attaining  some  goal.  In  general,  there 
are  many  goals  and  (possibly)  several  stereotypical  methods  of 
attaining  these  goals.  For  our  development,  these  scripts  are 
obtained  from  knowledge  of  the  enemy’s  doctrine  and  from  the 
experience  of  the  intelligence  experts  in  the  domain.  Actual 
sequences  of  reports  are  matched  against  these  scripts  in  order 
to  identify  those  scripts  that  best  explain  the  reports.  Special 
care  in  this  matching  is  required  to  account  for  the  temporal 
development  of  a  crisis.  The  selected  scripts  become  an  es¬ 
timate  of  the  likely  missions  and  of  the  plans  being  pursued. 
For  Schank,  the  reports  were  sentences  in  a  story;  in  our  case, 
intelligence  reports  in  an  unfolding  action. 

Although  script  matching  allows  the  elimination  of  many 
hypotheses,  the  variability  and  multiplicity  of  steps  and  se¬ 
quences  in  obtaining  a  goal  are  such  that  a  static  script  rep¬ 
resentation  is  not  adequate  for  detailed  understanding  of  re¬ 
alistic  situations.  For  example,  due  to  the  terrain  in  a  specific 
area,  the  ordering  of  troop  movements  and  the  length  of  time 
these  actions  take  could  be  considerably  different  them  what 
doctrine  specifies.  Planning  knowledge  must  be  introduced 
[Wil78,  Wil81,  SA77]  to  modify  and  expand  the  script  hy¬ 
potheses  as  a  situation  develops.  Thus,  by  using  script  match¬ 
ing  to  hypothesize  a  reduced  number  of  plausible  goals  for  the 
apparent  crisis,  it  becomes  feasible  to  generate  detailed  plans 
for  these  hypothesized  goals  using  the  observed  indicator  val¬ 
ues  and  the  current  situation  as  guides  for  the  available  op¬ 
tions.  The  plans  are  used  to  predict  future  observable  events 
that  could  discriminate  among  the  hypothesized  goals  1  Re¬ 
quests  for  further  critical  observations  to  refine  the  hypotheses 
can  then  be  generated  from  these  predictions.  The  reports, 
along  with  their  estimated  reliability,  are  matched  with  the 
hypothesized  plans  to  obtain  an  estimate  of  the  likelihood  of 
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each  plan.  The  plan  with  the  highest  likelihood  provides  the 
best  estimate  of  the  enemy’s  goal  and  future  actions. 

Background  and  Motivation 

PrevioTisIy,  MITRE  researchers  developed  ANALYST  [BonSl, 
LAB85],  EUi  expert  system  that  is  able  to  infer  real-time  situa¬ 
tion  displays  from  multiple  sensor  sources  and  also  to  process 
mission-oriented  information  requests.  ANALYST  answers 
these  requests  with  a  rule  base  of  static  critical  indicators 
that  refer  to  the  force  disposition.  ANALYST  is  also  part 
of  a  project  to  construct  a  set  of  cooperating  expert  systems 
called  ALLIES  [Bea86]  designed  to  perform  a  portion  of  the 
command  and  control  reasoning  process.  ALLIES  includes 
a  military  operations  planning  expert  system  (OPLANNER) 
[BDP86a]  and  a  object-oriented  simulation  of  the  war  (Bat¬ 
tlefield  Environment  Model)  (Nug84].  In  the  context  of  AL¬ 
LIES,  it  became  apparent  that  ANALYST  was  not  capable  of, 
but  had  the  potential  for,  in-depth  analysis  that  would  give 
a  clearer  picture  of  the  adversary’s  activities  and  intentions. 
Scripted  Analyst  (SCAN)  [LH87]  was  developed  to  monitor 
continuously  changing  situations  that  include  missing,  inac¬ 
curate  and/or  deceptive  sensor  data  and  to  analyze  multiple 
adversaries  while  at  the  same  time  preserving  the  desirable 
characteristics  of  a  rule-based  expert  system.  It  is  based  on 
a  script  representation  which  supports  a  heuristic  matching 
technique  to  determine  which  scripts  best  fit  a  set  of  reports. 

There  are  few  situation  monitoring  expert  systems  that 
have  been  developed  for  domains  as  volatile  as  our  Integrating 
Plans  and  Scripts  (IPS)  application.  In  [AFFH86,  AFHH87]  a 
plan  recognizer  with  a  simple  goal  detector  for  analyzing  air¬ 
craft  threat  is  described.  Blackboard  architectures  have  been 
used  in  domains  such  as  speech  processing,  but  these  systems 
do  not  fully  address  the  problems  that  an  expert  system  must 
handle  in  a  rapidly  changing  environment.  Fall’s  work  [Fal86] 
uses  a  representation  called  a  “model”  similar  to  scripts  to 
propagate  evidence  through  time  for  situation  monitoring,  but 
without  a  robust  interface  to  a  rule-based  expert  system.  The 
Ventilator  Manager  (VM)  program  [BS84]  is  an  example  of  a 
MYCIN-like  system  with  an  underlying  state  transition  model 
for  interpreting  data  in  an  intensive  care  unit,  but  was  found 
to  be  to  be  inadequate  for  monitoring  data  continuously  over 
time.  The  work  of  Kautz  and  Allen  [KA86]  is  a  bottom  up 
plan  recognizer  (i.e.  it  tries  to  discover  all  plans  that  con¬ 
tain  the  reports)  and  in  our  domain  would  be  combinatorially 
explosive. 

MITRE  has  been  working  on  AI  techniques  for  military 
planning  applications  since  1979  and  developed  the  KBS  sys¬ 
tem  in  1980.  KBS  was  applied  to  joint  crisis  response  planning 
in  support  of  the  Ofiice  of  the  Joint  Chiefs  of  Staff.  The  KBS 
system  employed  a  powerful  but  difficult  to  engineer  proce¬ 
dural  model  of  planning.  This  inflexibility  led  the  MITRE 
researchers  to  consider  planning  from  a  rule-based  perspec¬ 
tive.  The  OPLANNER  system  is  an  investigation  into  the  use 
of  hierarchical  planning  techniques  controlled  by  rules  em¬ 
ploying  constraints  within  and  among  portions  of  the  plan. 
OPLANNER  is  more  a  framework  than  a  system,  in  that  it 
is  composed  of  a  network-based  rule  interpreter,  knowledge 
structures  and  metarplanning  strategies,  and  a  small  set  of 
example  planning  operators  tailored  to  two  separate  planning 
domains.  It  has  been  applied  in  the  past  to  joint  opera¬ 
tions  crisis  response  planning  [BDP86a]  and  to  a  more  tac¬ 
tical  ground  battle  planning  scenario  [Bea86],  An  improved 
system,  REDPLANNER,  was  developed  to  generate  the  de¬ 
tailed  plans  required  for  IPS.  While  the  REDPLANNER  sys¬ 
tem  employs  the  OPLANNER  rule  interpreter  and  many  of 
the  planning  lessons  learned  during  the  earlier  OPLANNER 
efforts  [BDP86b],  the  use  of  DeKleer’s  ATMS  [dK84,  dKSfi] 


concepts  for  maintaining  knowledge  about  multiple  plan  al¬ 
ternatives  is  a  major  enhancement  of  the  OPLANNER  AI 
planning  framework. 

Approach 

In  a  previous  section  we  have  discussed  the  motivating  in¬ 
fluence  of  semantic  analysis  of  natural  language.  However, 
the  analogy  to  natural  language  understanding  should  not  be 
carried  too  far.  There  are  several  features  of  the  intelligence 
plan  recognition  problem  that  are  not  significantly  present  in 
natural  language  understanding: 

•  Most  events  have  a  characteristic  duration  that  is  signif¬ 
icant  in  the  recognition  of  plans. 

•  It  appears  to  be  necessary  to  carry  forward  multiple  hy¬ 
potheses  and  to  reevaluate  previously  processed  reports. 

•  Large  numbers  of  unrelated  reports  are  received. 

•  The  reports  (not  just  their  interpretations)  have  inherent 
associated  errors  and  ambiguities. 

•  The  reports  are  from  different  sensors  and  thus  have  dif¬ 
ferent  significance. 

The  approach  described  below  was  motivated  by  these  issues. 

The  basic  structure  of  the  IPS  system  is  shown  in  Fig¬ 
ure  1.  SCAN  matches  the  reports  against  a  set  of  static  scripts 
in  order  to  suggest  a  number  of  possible  enemy  goals.  We  as¬ 
sume  multisensor  correlation  prior  to  IPS  by  using  only  pro¬ 
cessed  reports;  that  is,  the  reports  are  of  units  and  their  ac¬ 
tivity  (for  example,  motorized  infantry  moving  west)  not  of 
sensor  data.  This  simplification  was  necessary  to  limit  the 
scope  of  the  IPS  project.  SCAN  reasons  about  event  dura¬ 
tions  and  time  relations  between  events,  but  does  not  repre¬ 
sent  turning  points  (alternative  sequences  of  events  within  a 
script).^  When  SCAN  determines  that  a  script  (or  scripts) 
has  been  matched,  it  passes  on  the  script  goal  for  a  specific 
context  (locations  and  time)  and  the  related  reports  to  the 
REDPLANNER.  REDPLANNER  uses  the  goals  and  reports 
from  SCAN  to  generate  alternative  plans  for  attaining  the 
goals.  While  the  scripts  are  stereotypical  and  static,  the  plans 
generated  account  for  the  current  situation  and  alternatives  in 
attaining  the  goals.  REDPLANNER  is  a  hierarchical  planner 
using  modified  procedural  networks  to  represent  the  plans.  In 
order  to  efficiently  generate  and  represent  many  alternative 
plan  structures,  the  procedural  networks  were  augmented  by 
some  of  the  techniques  of  Assumption-based  Truth  Mainte¬ 
nance  Systems  (ATMS)  [dK84,  dK86].  The  alternative  plans 
for  the  goals  hypothesized  by  SCAN  are  analyzed  by  the  Plan 
Analyzer. 

The  Plan  Analyzer  compares  the  plans  with  each  other 
and  with  the  reports  in  order  to  rank  them  as  estimates  of 
the  situation.  It  also  searches  for  predicted  events  the  can 
be  observed  and  that  discriminate  among  the  plans.  If  these 
events  are  observed,  pruning  the  set  of  hypothesized  plans 
and  goals  is  possible.  The  discriminating  predicted  events  are 
passed  to  SCAN  as  cues  for  its  monitoring  and  matching  of 
incoming  reports. 

Development  and  testing  of  IPS  is  done  with  simulated 
data.  Since  the  IPS  system  reasons  about  causal  relation¬ 
ships  among  events,  the  simulated  events  must  appear  to  be 
causally  related.  The  simulation,  Soviet  Tactical  Forces  En¬ 
vironment  Model  (STFEM)^,  specially  constructed  for  IPS, 


*  Alternative  sequences  are  dealt  with  in  the  planning  phase. 
®STFEM  wais  designed  and  iinplemented  by  John  R.  Davidson  and 
Russell  R.  Leighton  of  the  MITIiLE  Corporation. 
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Figure  1:  IPS  Architecture 


produces  these  events  by  using  scripts  to  represent  the  sim¬ 
ulation  scenarios.  These  scripts  (which  are  slightly  different 
from  the  SCAN  scripts)  describe  events  in  their  causal  rela¬ 
tionships  for  expansion  by  the  simulator. 

The  following  sections  provide  additional  descriptions  of 
these  four  parts  of  the  IPS  system. 

SCAN 

Prom  a  set  of  reports  based  on  behaviors  generated  by 
STFEM,  SCAN  recognizes  and  monitors  trends  in  activities 
as  they  unfold  and  passes  its  evaluation  of  the  current  sta¬ 
tus  onto  the  REDPLANNER  for  further  refinement  as  shown 
in  Figure  1.  These  reports  from  STFEM  include  informa¬ 
tion  about  units,  their  size  and  type,  direction  of  movement 
and  level  of  activity.  It  is  assumed  that  the  problem  of  coa¬ 
lescing  information  from  multiple  sources  is  managed  at  the 
simulation  end.  Unlike  ANALYST,  SCAN  deals  only  with  the 
higher  level  information  and  with  scenario  detection  and  not 
with  data  fusion.  SCAN  searches  a  set  of  scripts  to  determine 
those  which  are  plausible  explanations  of  the  current  activ¬ 
ity,  that  is,  which  scripts  have  a  strong  likelihood  of  matching 
against  the  observed  behavior. 

The  script  knowledge  base  is.  stored  as  a  taxonomy  of 
indicators  which  describe  stereotypical  behavior.  A  SCAN 
script  (see  Figure  2,  which  is  not  necessarily  indicative  of 
true  doctrine)  includes  a  list  of  events  called  script-elements 
such  as  increased-iogjsf/cs-and-servjce-actiVifies.  Time  rela¬ 
tions  between  script-elements  are  expressed  by  the  order¬ 
ing  predicates.  For  example,  increased-reconsJssance  occurs 
during  increased-logistic-and-servjce-activitjes  with  the  for¬ 
mer  beginning  approximately  60  minutes  after  the  latter  has 
started.  The  time  formalisms  are  based  on  work  by  James 
Allen  [A1184,  AH85].  Script  elements  may  be  primitive  indi¬ 
cators  called  propositions  (such  as  increased-reconaissance)  or 
they  may  point  to  other  scripts  in  the  hierarchy.  The  subscript 
attack-with-fank-div  starts  approximately  120  minutes  after 
mcreased-reconaissajice  starts  and  may  continue  after  it  ends. 
Either  an  attack-with-tank-div  or  an  attack-with-motorized-' 
riSe-div  could  be  occuring  in  a  frontal  attack,  therefore,  both 
are  monitored,  bindings  are  passed  to  the  script.  They  de¬ 
scribe  the  context  of  the  instantiation  of  a  script  or  script- 
element  (e.g.  location  and  time)  and  the  default  constants 
such  as  the  duration  of  an  event.  The  preconditions  specify 
entry  conditions  for  the  script  to  be  a  candidate  hypothesis. 
They  are  divided  into  necessary  and  sufficient  preconditions 
based  on  whether  the  condition  is  static  or  dynamic.  For  ex¬ 
ample,  a  frontal  area  is  a  necessary  condition  for  a  frontal 
attack  as  represented  in  Figure  2;  while  in  another  script,  the 
presence  of  three  tank  battalions  could  be  a  sufficient  precon¬ 
dition  for  an  attack  by  a  tank  division. 


(defscript  frontal- attack 

(script-name  frontal-attack) 

(script-elements 

(increased-logistic-and-service-activities 

proposition 

(> location  >time  (>  duration  inhnite))  nil) 
(increased-recon  aissance 
proposition 

(>  location  >time  (>duration  240)) 

(during  increased-iogisfic-and-service-activities  60)) 
( attack-  with-  tank-  div 
script 

(>Jocation  >time) 

(overlaps  increased-reconaissance  120)) 
(attack-with-motorized-riSe-div 
script 

(>  location  >timej 

(overlaps  increased-reconaissance  120)))) 

(bindings  (>  location  >time)) 

(necessary-preconditions  (frontal-areaj ) 
(sufficient-preconditions  t) 

(script-analysis  max-script-likelihood)) 

Figure  2:  Simplified  Frontal  Attack  Script 

The  first  step  in  determining  which  scripts  are  applicable 
to  the  observed  behavior  is  to  find  those  scripts  with  satis¬ 
fied  pre-conditions  under  the  current  context  (time  and  Ibcar 
tion).  The  underlying  primitive  events  are  extracted  from  the 
resulting  subset  of  scripts.  SCAN  then  applies  rules  about 
what  situations  contribute  to  the  existence  of  the  primitive 
events  in  order  to  determine  the  likelihood  of  each  primitive 
event  occurring.  An  overall  likelihood  is  associated  with  the 
scripts  based  on  the  likelihoods  of  their  constituent  primitive 
events  and  the  time  relations  between  those  events.  The  pro¬ 
cess  of  determining  the  script  likelihood  is  complex  because 
it  must  take  into  consideration  the  ordering  and  duration  of 
events  as  well  as  missing  events.  Data  on  the  events  are  de¬ 
rived  from  reports  which  are  often  imprecise.  In  addition,  the 
activities  themselves  might  not  follow  the  stereotypical  script 
exactly.  To  determine  the  overall  script  likelihood,  SCAN  as¬ 
signs  a  likelihood  to  each  script  element  (up  to  the  one  for  the 
ciirrent  time)  by  multiplying  the  old  script  element  likelihood 
by  the  percentage  of  sequencing  preconditions  and  duration 
information  that  match  the  reports.  This  likelihood  can  be 
farther  adjusted  by  weighting  it  according  to  the  importance 
of  that  element  in  the  script.  Then,  an  average  of  all  these 
likelihoods  is  computed  to  arrive  at  the  script  likelihood.  If 
the  resulting  script  likelihood  4s  high,  then  the  script  is  con- 
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sidered  to  be  active.  As  time  goes  on,  the  script  likelihood 
will  either  stay  high  enough  for  the  script  to  remain  active, 
or  the  likelihood  will  decay  as  script  elements  are  missed  or 
occur  out  of  sequence. 

The  active  scripts  at  any  given  point  in  time  represent  a 
set  of  hypotheses  of  the  enemy’s  intentions.  This  list  of  ac¬ 
tive  scripts  and  their  associated  bindings  are  passed  on  to  the 
REDPLANNER  for  further  consideration.  REDPLANNER 
takes  the  candidate  scripts  and  goes  through  a  process  of  fur¬ 
ther  refinement  and  generation  of  multiple  plans.  Even  with 
a  constrained  set  of  candidate  scripts,  the  number  of  possi¬ 
ble  scenarios  when  expanded  may  be  very  large.  Therefore, 
the  planner  may  send  queries  back  to  SCAN.  These  queries 
can  include  questions  about  the  disposition  of  units  in  a  par¬ 
ticular  location  at  a  given  time.  In  addition,  the  Plan  An¬ 
alyzer,  which  evaluates  plan  consistency,  may  suggest  other 
hypothesized  missions  for  SCAN  to  monitor.  A  more  detailed 
description  of  SCAN  may  be  found  in  [LH87]. 

REDPLANNER 

REDPLANNER  is  a  rule-based,  hierarchical  planner  using 
constraint  propagation  to  limit  the  search  for  plausible  al¬ 
ternatives  [Sac77,  SteSl,  Wil84,  BDP86a].  Plans  and  their 
abstractions  are  represented  in  procedural  networks  [Sac77]. 
Procedural  knowledge  in  the  nodes  of  the  network  is  repre¬ 
sented  by  rules.  Constraints  consist  of  dynamic  typing  of  the 
plan  variables  and  of  rules  relating  multiple  plan  variables. 
Figure  3  shows  the  hierarchical  structure  of  a  plan.  In  this  ex¬ 
tremely  simplified  example,  the  red  mission  is  an  ATTACK 
which  has  been  decomposed  into  three  phases,  ASSEMBLE, 
DEPLOY,  and  STRIKE.  In  a  more  detailed  representation, 
each  of  the  sub-plans  would  be  further  decomposed  into  sub¬ 
plans  such  as  TRANSPORT,  MANEUVER,  etc. 

Alternative  plans  and  goals  are  represented  by  incor¬ 
porating  the  techniques  of  Assumption-based  Truth  Mainte¬ 
nance  Systems  (ATMS)  [dK86]  into  the  procedural  network. 
Whenever  the  constraints  cannot  eliminate  all  but  one  alter¬ 
native,  the  planning  system  must  make  choices.  These  choices 
are  represented  as  assumptions  in  the  ATMS  structure.  Al¬ 
ternatives  are  mutually  inconsistent  choices.  Thus,  a  given  al¬ 
ternative  plan  is  described  by  the  set  of  choices  one  believes — 
automatically  excluding  those  choices  inconsistent  with  the 
current  beliefs.  Another  plan  is  described  by  a  different  set  of 
choices.  The  rule  evaluation  mechanism  ensures  that  the  vari¬ 
ables  of  the  rules  are  consistent  so  that  only  inferences  based 
on  consistent  sets  of  assumptions  are  made. 

The  function  of  REDPLANNER  is  to  generate  feasible 
plans  for  the  hypothesized  missions  received  from  SCAN. 
These  plans  must  be  consistent  with  the  reports  and  the  cur¬ 
rent  situation.  SCAN  will  have  passed  on  the  reports  used 
to  match  the  script  along  with  the  script  context  identifica¬ 
tions  it  has  made.  Recognizing  that  the  reports  are  inaccurate 
and  that  SCAN  has  had  to  force  them  into  a  finite  number 
of  stereotypical  scripts,  REDPLANNER  uses  the  reports  as¬ 
sociated  with  specific  plan  elements  only  as  guidance  in  plan 
generation.  The  plans  generated  loosely  follow  red  doctrine 
and  the  alternatives  selected  are  those  most  closely  matching 
the  reports.  In  general,  the  reports  are  ambiguous  and  have 
likelihoods  associated  with  them;  the  selection  of  plan  alter¬ 
natives  takes  these  likelihoods  into  account.  In  addition  to 
allowing  flexible  selection  of  alternatives,  the  use  of  planning 
allows  the  analysis  to  be  constrained  by  the  current  situation; 
that  is,  the  political  situation,  the  local  terrain  and  weather, 
the  resources  available,  and  the  possible  counter  actions  all  af¬ 
fect  the  actions  likely  to  occur.  REDPLANNER  will  consider 
these  factors  in  generating  plan  alternatives. 

Even  with  the  report  and  situational  constraints,  the 
number  of  possible  plans  is  large.  To  help  in  limiting  what 


is  a  combinatorial  explosion,  not  all  plans  are  expanded  com¬ 
pletely.  The  resulting  partial  plans  include  significant  past 
and  near  term  observable  events  but  do  not  include  less  sig¬ 
nificant  details  or  many  alternatives  into  the  future.  As  re¬ 
ports  arrive  and  time  passes  these  partial  plans  are  pruned, 
expanded,  and  modified  to  maintain  consistency  and  currency. 

This  may  be  illustrated  by  an  example.  Figure  4  shows 
an  idealized  battle  area  roughly  modeled  after  the  Fulda  Gap. 
The  scenario  in  this  example  is  a  Red  attack  on  Bad  Hersfeld 
as  shown  in  Figure  4,  The  first  indication  REDPLANNER 
hzis  of  this  activity  occurs  during  the  initial  movements  to¬ 
ward  the  assembly  area  when  SCAN  suggests  the  possibility 
of  attack  (among  other  missions  such  as  force  replacement) 
in  the  general  Fulda-Bad  Hersfeld  area.  This  suggestion .  is 
based  on  reports  that  indicate  a  western  movement  of  combat 
forces  in  the  area  of  Werra  as  illustrated  in  Figure  5.  At  this 
point,  REDPLANNER  could  generate  detailed  plans  for  an  at¬ 
tack,  Since  there  are  many  possible  objectives,  deployments, 
assembly  areas,  etc.,  many  alternative  plans  would  be  devel¬ 
oped.  This  is  inefficient,  confusing,  and  unnecessary.  With 
this  paucity  of  information,  REDPLANNER  generates  only  a 
partial  plan  based  on  movements  in  general  areas  and  with¬ 
out  detailed  unit  planning.  This  plan  is  like  that  represented 
in  Figure  3  in  that  it  considers  only  mass  troop  movements 
between  general  areas.  Figure  6  shows  the  areas  considered. 
The  abstract  plan  estimates  duration  and  general  directions 
of  the  the  movements.  As  additional  reports  arrive  and  a  bet¬ 
ter  indication  of  the  Red  activity  is  obtained,  more  specific 
objectives  and  movements  are  planned. 

Plan  Analyzer 

The  basic  function  of  Plan  Analyzer  is  to  identify  the  most 
likely  mission  and  the  plan  being  executed  to  achieve  that  mis¬ 
sion.  In  order  to  do  this,  it  evaluates  the  generated  plans  for 
consistency  with  the  reports,  for  general  feasibility,  for  sus¬ 
ceptibility  to  counteraction,  for  consistency  with  the  global 
situation,  etc.  As  new  reports  arrive,  they  are  used  for  plan 
re-evaluation.  The  analysis  of  the  alternative  plans  supplied 
by  REDPLANNER  are  closely  integrated  with  the  actual  plan 
development  since  the  analysis  may  suggest  additional  alter¬ 
natives  or  additional  expansion  of  some  aspect  of  a  plan  in 
order  to  compare  with  new  reports  or  to  compare  plans.  An¬ 
other  function  of  Plan  Analyzer  is  the  prediction  of  observable, 
discriminating  events  to  assist  in  the  evaluation  and  ultimate 
pruning  of  plans.  To  do  this,  plan  differences  need  be  discov¬ 
ered  and  knowledge  of  sensor  capabilities  is  required.  These 
predictions  are  passed  to  SCAN  so  that  it  will  monitor  the 
reports  for  related  information.  Later  development  of  Plan 
Analyzer  will  include  passing  collection  requests  to  the  simu¬ 
lated  sensors. 

A  powerful  future  function  of  Plan  Analyzer  is  to  recog¬ 
nize  repeated  plans  that  do  not  have  associated  SCAN  scripts. 
This  means  that  a  previously  unknown  method  of  achieving 
some  goal  has  been  discovered.  This  could  detect  doctrinal 
changes  as  well  as  omissions  and  errors  in  the  script  defi¬ 
nitions.  Having  made  such  a  recognition,  the  next  step  is 
to  generate  a  new  script  for  SCAN.  This  requires  extensive 
knowledge  of  script  structure  and  semantics. 

Since  the  Plan  Analyzer  must  deal  with  the  procedural 
network  representation  of  the  alternative  plans,  it  will  be  im¬ 
plemented  as  an  asynchronous  phase  of  the  planning  system. 
As  mentioned  above,  the  planning  system  is  rule  driven.  It 
will  be  necessary  to  add  a  network  matching  capability  to  this 
system  in  order  to  perform  the  plan  analysis. 

STFEM 

STFEM  is  an  object-oriented,  script-driven,  scenario  gener¬ 
ation  tool  that  provides  a  directed  set  of  sensor  reports  to 
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Figure  3:  A  Simple  Plan 


FEBA 


feed  the  PPS  intelligence  support  system.  It  builds  on  work  at 
MITRE  in  advanced  behavioral  simulations  [Nug84,  MK82]. 
Simulation  software  tends  to  be  complex  and  highly  focused 
for  a  particular  domain  application.  When  used  to  supply 
ground  truth  for  intelligence  support  systems,  such  simula¬ 
tions  are  often  difficult  to  alter  or  extend.  STFEM  was  de¬ 
veloped  as  a  scenario  generation  tool  which,  while  providing 
a  suitably  rich  environment  for  a  variety  of  intelligence  func¬ 
tions,  would  also  allow  users  to  create  these  scenarios  in  a  flex¬ 
ible  way.  User  interaction  with  STFEM  is  based  on  defining 
and  instantiating  a  series  of  spacial  templates  and  procedu¬ 
ral  scripts  which  form  the  basis  for  the  scenario.  The  current 
situation  is  an  attack  in  the  Fulda  Gap  area  of  Germany.  In 
this  case  STFEM  begins  with  a  set  of  default  attack  scenar¬ 
ios.  Throughout  the  scenario  definition  process,  the  user  can 
simply  choose  a  certain  default  configuration,  alter  a  given 
configuration,  or  define  a  new  scenario  within  the  structure  of 
a  very  high  level  attack  script. 

To  create  a  STFEM  scenario  for  the  current  situation,  a 
user  first  chooses  one  from  a  set  of  available  attack  scripts. 
Each  attack  script  describes  a  road  march  from  a  garrison 
^ea  through  assembly  and  deployment  areas  to  a  target  that 
is  selected  by  the  user.  STFEM  currently  supports  pincer, 
envelopment,  and  flank  attack  scenarios.  While  each  script 
defines  basic  deployment  and  attack  plans  and  configurations, 


STFEM  allows  the  user  to  choose  the  route  and  tactical  area 
locations  as  well  as  the  military  units  that  compose  the  at¬ 
tack  force.  Only  these  three  scencirios  are  currently  supported; 
however,  the  STFEM  scripts  can  easily  be  augmented  to  sup¬ 
port  any  number  of  scenarios.  These  scenarios,  in  turn,  can  be 
used  to  provide  a  rich  set  of  sensor  reports  to  other  intelligence 
support  systems. 

The  scenarios  are  single-sided  and  concentrate  on  unit 
movement  and  communication.  STFEM  generates  observ¬ 
ables  in  an  aggregated  sense.  Rather  than  simulate  each  event 
such  as  a  radar  emission  or  communications  message,  it  mod¬ 
els  the  level  of  activity  over  time  for  each  of  five  observable 
phenomena.  These  are:  comunications  activity,  radar  activity, 
motion,  indirect  fire,  and  optical  visibility.  Each  unit  involved 
in  a  particular  phase  of  the  operation  is  assigned  a  level  repre¬ 
senting  the  rate  of  movement,  communications,  radar  activity, 
etc.,  for  that  time  period.  These  histograms  of  activity  are  car¬ 
ried  in  the  object  structures  for  each  actor  as  the  actor  pursues 
the  assigned  military  goals.  They  are  contextually  derived  and 
form  one  of  the  bases  that  SCAN  and  REDPLANNER  use  in 
computing  the  likelihoods  of  events, 

STFEM  sensors  are  tasked  by  regions  that  act  as  areas  of 
focus.  In  this  way  we  can  task  the  system  to  collect  more  intel¬ 
ligence  signals  in  a  particular  area  or  “fly”  an  imagery  mission 
in  another  area  without  individually  modelling  the  instances 
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Figure  5:  Initial  Reports 


of  those  sensors.  Each  region  has  a  level  of  “acuity”  for  each 
of  its  observable  classes.  This  acuity  is  the  probability  and 
accuracy  with  which  the  report  is  made.  The  acuity  associ> 
ated  with  each  sensor  is  also  used  to  provide  an  error  measure 
that  may  be  used  as  an  indication  of  the  confidence  in  the 
report.  This  is  an  acknowledged  approximation  of  tactical 
sensor  suites  that  would  not  be  appropriate  in  other  contexts, 
but  serves  well  in  the  IPS  system. 

Accomplishments 

The  IPS  project  is  a  little  more  than  half  complete.  An  initial 
demonstration  system  containing  all  the  parts  shown  in  Figure 
1  has  been  built,  but  the  full  functionality  described  above  is 
not  yet  available. 

The  scenarios  used  for  the  STFEM  simulation  are  ori¬ 
ented  towards  ground  activities  and  include  flank,  frontal  and 
pincer  attacks.  These  unclassified  scenarios  are  suitable  for 
the  proof  of  concepts  that  will  have  a  wider  domain  of  ap¬ 
plicability.  The  STFEM  tools  are  completed,  but  more  and 
richer  scenarios  will  be  needed  for  future  research. 

SCAN  has  new  scripts  that  stereotypically  model  several 
missions,  including  those  of  the  simulated  scenarios.  Simple 
spatial  template  matching  is  used  to  detect  force  deployment. 
When  SCAN  receives  some  reports  from  the  simulated  sen¬ 
sors  and  assigns  one  or  more  missions  to  that  activity,  these 
and  the  sensor  reports  are  passed  on  to  RED- 

PLANNER. 

REDPLANNER  incorporates  augmented  ^sumption- 
based  Truth  Maintenance  System  (ATMS)  techniques  in  or¬ 
der  to  facilitate  the  generation  of  alternative  and  contingency 
plans.  When  REDPLANNER  receives  hypothesized  missions 
and  sensor  reports  from  SCAN,  it  generates  a  number  of  al¬ 
ternative  plans  based  on  the  mission  assignments,  the  sensor 
reports  and  red  doctrine. 

The  Plan  Analyzer  is  partially  developed.  It  performs  a 
limited  comparison  and  evduation  of  plans  based  on  doctrine 
and  new  sensor  input.  It  also  predicts  events,  but  the  ability 
to  detect  events  that  are  discriminatory  is  limited. 

The  demonstration  system  includes  two  innovative  ad¬ 
vancements  of  existing  techniques.  The  first  involves  tempos 


ral  reasoning  about  durations  of  action  that  are  of  uncertain 
length.  Scripts  express  doctrinal  or  experiential  knowledge 
about  event  timing.  In  practice,  the  time  required  to  complete 
an  event  (such  as  movement)  may  vary  considerably  from  ac¬ 
tion  to  action.  This  will  cause  the  time  between  reports  of 
sequential  events  to  deviate  significantly  from  the  stereotypi¬ 
cal  value.  It  is  insufficient  to  test  only  the  sequence  of  events 
since,  in  general,  the  timing  of  events  is  critical  to  proper 
script  matching.  For  these  reasons,  the  temporal  reasoning 
development  included  a  flexible  time  interval  matching  proce¬ 
dure. 

The  second  innovation  was  the  development  of  a  represen-  . 
tation  for  alternative  plans  in  a  hierarchical  planning  system. 
Since  there  may  be  several  possible  missions  and  many  ways 
to  achieve  these  missions,  even  within  the  constraints  of  the 
situation  and  reports,  it  was  critical  that  a  conceptually  per¬ 
spicuous  method  for  describing  and  comparing  these  plans  be 
developed.  It  was  also  necessary  to  develop  a  computationally 
efficient  technique  for  developing  these  plans.  The  techniques 
of  ATMS,  which,  in  effect,  qualify  every  part  of  the  plan  rep¬ 
resentation  by  its  plan  alternative,  were  enhanced  to  provide 
the  required  representation  and  control. 

Future  Work 

The  number  of  scenarios  and  indicators  must  be  expanded 
and  the  inference  rules  improved  to  demonstrate  the  opera¬ 
tional  feasibility  of  IPS.  Further  development  of  the  compar¬ 
ison  of  plans  and  the  prediction  of  discriminating  events  is 
required.  A  more  formal  planning  language  is  necessary  to 
enhance  plan  comparison  through  the  use  of  partial  plans,  de¬ 
ferred  plan  completion,  and  replanning.  Mechanisms  for  the 
feedback  of  new  scripts  as  well  as  sensor  report  predictions 
must  be  included  to  increase  the  sensitivity  and  selectivity  of 
SCAN. 

References 

[AFFH86]  J.  Azarewicz,  G.  Fala,  R.  Fink,  and  C.  Heithecker. 

plan  recognition  for  airborne  tactical  decision¬ 
making.  In  AAAI,  pages  805-811,  1986, 


451 


Figure  6:  Abstract  Attack  Plan 


[AFHH87]  J.  Azarewicz,  G.  Fala,  I.  Hayslip,  and  C.  Hei- 
thecker.  Mulit-agent  plan  recognition  in  an  ad¬ 
versarial  domain.  In  ESIG,  pages  188-193,  1987. 

[AH85]  J.  Allen  and  P.  J.  Hayes.  A  common  sense  theory 
of  time.  In  IJCAI,  pages  528-531,  1985. 

[A1184]  J.  Allen.  Towards  a  general  theory  of  actions  and 
time.  Artificial  Intelligence,  23:123-154,  1984. 

[BDP86a]  J.  W.  Benoit,  J.  R.  Davidson,  and  E.  G.  Powell. 

Artificial  intelligence  in  joint  deployment  planning. 
In  Artificial  Intelligence  for  Military  Applicaiions, 
ORSA,  Arlington,  VA,  October  1986. 

[BDP86b]  J.  W.  Benoit,  J.  R.  Davidson,  and  E.  G.  Powell. 

Lessons  learned  developing  a  planning  system.  In 
Proceedings  of  the  1st  International  Conference  on 
Applications  of  Artificial  Intelligence  m  Engineer¬ 
ing  Problems,  Springer- Ver lag,  Southampton  Uni¬ 
versity,  U.  K.,  April  1986. 

[Ben86]  J.  W.  Benoit,  et  al.  An  experiment  in  coop¬ 
erating  expert  systems  for  command  and  con¬ 
trol.  In  Expert  Systems  in  Government  Confer¬ 
ence,  pages  372-380,  October  1986. 

[BonSl]  R.  P.  Bonasso.  An  expert  system  for  processing 
sensor  returns.  In  The  First  Army  Conference  on 
Knowledge-Based  Systems  for  C$1,  pages  219-245, 
Ft.  Leavenworth,  November  1981. 

[BS84]  B.  G.  Buchanan  and  E.  H.  Shortliffe.  Rule-Based 
Expert  Systems.  Addison- Wesley  Publishing  Co, 
1984. 

[dK84]  Johan  de  Kleer.  Choices  without  backtracking.  In 
Proceedings  Fourth  National  Conference  on  Artifi¬ 
cial  Intelligence,  pages  79-85,  Austin,  TX,  1984. 

[dK86]  Johan  de  Kleer.  An  assumption-based  TMS.  Ar¬ 
tificial  Intelligence,  28:127-162,  1986. 

[Fal86]  T.  C.  Fall.  Evidential  reasoning  with  temporal 
aspects.  In  AAAI,  pages  891-895,  1986. 

[KA86]  H.  A.  Kautz  and  J.  F.  Allen.  Generalized  plan 
recognition.  In  AAAI,  pages  32-37,  1986. 


[LAB85]  S.  J.  Laskowski,  H.  J.  Antonisse,  and  R.  P. 

Bonasso.  ANALYST  II:  a  knowledge-based  intelli¬ 
gence  support  system.  In  Second  IEEE  Conference 
on  Artificial  Intelligence  Applications,  pages  552- 
563,  December  1985. 

[LH87]  S.  J.  Laskowski  and  E.  J.  Hofmann.  Script-based 
reasoning  for  situation  monitoring.  In  AAAI, 
1987. 

[MK82]  D.  McArthur  and  P.  Klahr.  The  ROSS  Language 
Manual.  Technical  Report  N-1884-AF,  RAND 
Corporation,  1982. 

[Nug84]  R.  O.  Nugent.  MITRE’s  Battlefield  Environment 
Model.  In  ORS A/TIMS  Conference,  November 
1984.  Working  Group  Presentation. 

[SA77]  R.  C.  Schank  and  R.  P.  Abelson,  Scripts,  Plans, 
Goals  and  Understanding.  Lawrence  Erlbaum  As¬ 
sociates,  New  York,  NY,  1977. 

(Sac77]  E.  D.  Sacerdoti.  A  Structure  for  Plans  and  Behav¬ 
ior.  Elsevier  North-Holland,  Inc.,  1977. 

[SteSl]  M.  Stefik.  Planning  with  constraints  (MOLGEN: 

part  1).  AHificial  IrUelligence,  16:111-140,  1981. 

[Wil78]  R.  Wilensky.  Understanding  Goal-based  Stories. 
PhD  thesis,  Yale  University,  1978. 

[Wil81]  R.  Wilensky.  PAM.  In  R.  C.  Schank  and  C.  K. 

Riesbeck,  editors.  Inside  Computer  Understand¬ 
ing,  chapter  7,  Lawrence  Erlbaum  Associates, 
Hillsdale,  NJ,  1981. 

[Wil84]  D.  E.  Wilkins.  Dommn-independent  planning: 

representation  and  plan  generation.  Artificial  In¬ 
telligence,  22:269^301,  1984. 


452 


APPENDIX  A 

REGISTERED  DFS-88  ATTENDEE  LIST 


A-1 


THIS  PAGE  LEFT  BLANK  INTENTIONALLY 


A- 2 


nFS-88  ATTENDEE  LIST 


LAST  NAME  FIRST  NAME  COMPANY 


Abbott 

Dr.  Ned  E. 

Abbott 

Dr.  Richard  J. 

Abend 

Mr.  Kenneth 

Adai  r 

Mr.  Sidney  T. 

Adams 

Mr.  Neil  J. 

Agnello 

Ms.  Jennie  M. 

Aj  mera 

Dr.  Ramesh  C. 

A1  banese 

Mr.  Anthony  J. 

Alder  son 

Ms.  Susan  L. 

Alexander 

Dr.  Lawrence  D. 

A1  len 

Mr.  Robert  W. 

Alter 

Ms.  Donna  P. 

Anderson 

Dr.  Stephen  L. 

Anderson 

Mr.  Kenneth  S. 

Annen 

Mr.  Robert 

Annen 

Mr.  Robert  P. 

Antoni k 

Mr.  Joseph 

Antony 

Mr.  Richard  T. 

Armstrong 

Mr.  John  E. 

Askin 

Dr.  Kurt 

Attil  i 

Mr.  Michael  A. 

Babcock 

Dr.  James  H. 

Baer 

Mr.  Richard  T. 

Bailey 

Mr.  Robert  E. 

Baran 

Mr.  Robert  H. 

Barnes 

Dr.  Christopher 

Barnett 

Mr.  Paul  G. 

Barr 

Mr.  John  K. 

Barrett 

Dr.  Terence  W. 

Bartu sek 

Mr.  Donald  L. 

Batchelder 

Mr.  Charles 

Baxter 

Mr.  Sam  E. 

Beamer 

Mr.  Walter  C. 

Beavers 

CDR  Michael  C. 

Becker 

Mr.  Daniel 

Belkin 

Dr.  Barry 

Bendekovic 

Mr.  John  P. 

Benedict 

Mr.  Richard  D. 

Benites 

Mr.  Ernesto 

Bennett 

Mr.  Steven 

Bent 

Mr.  Nathaniel  E 

Berman 

Dr.  Arie 

Bernardin 

Dr.  Charles  P. 

Bertapelle 

Mr.  Ken  A. 

Betts 

Mr.  Roger  S. 

Betz 

Mr.  John 

Betz 

Mr.  John  W. 

Bianchi 

Ms.  April 

Billig 

Mr.  Lewis  S. 

Bogart 

Dr.  Elliot 

Bowen 

Dr.  Julius  I. 

E-Sy stems,  Inc. /Garland  Div. 

Lockheed  Aeronautical  Systems  Co. 
Inter spec,  Inc. 

Flam  &  Russell  Inc. 

Loral  (Defense  Systems  Division) 

D.  H.  Wagner,  Associates 
Naval  Weapons  Center 
SENSIS  Corporation 
NOSC 

General  Electric 
Boeing  Aerospace  Co. 

E-Systems,  Inc./Melpar  Div. 

D.  H.  Wagner,  Associates 
Locus,  Inc. 

ITT  Avionics  Division 
ITT  Avionics 

Rome  Air  Development  Center 
USA  Ctr.  for  Signals  Warfare 
GTE-Gov't.  Systems  Corp. 

NRL 

M/A-COM,  Inc. 

The  MITRE  Corporation 
ODASD(I) 

UNISYS 

Naval  Surface  Warfare  Center 
Los  Alamos  National  Laboratory 
Johns  Hopkins  Univ. 

The  MITRE  Corp. 

W.  J.  Schafer  Associates 

Naval  Surface  Warfare  Center 

Grurrman  Data  Systems 

General  Dynamics/Fort  Worth  Division 

Naval  Air  Development  Center 

OPNAV 

Naval  Air  Development  Center 
Daniel  H.  Wagner  Assocs. 

System  Planning  Corp. 

Litton  Data  Systems 
Naval  Weapons  Center 
The  MITRE  Corp 

Decision-Science  Applications,  Inc. 
Hughes  Aircraft  Co. 

Texas  Instruments 
McDonnell  Douglas 
Analytics,  Inc. 

TASC 

TASC 

Martin  Marietta  Info.  &  Comm.  Sys. 

Rand  Corp 

TRW 

Johns  Hopkins  University  APL 


PHONE  NUMBER 

214- 272-0515 
818-847-1455 

215- 834-1511 

215- 674-5100 

216- 796-9885 
202-767-3882 
619-939-3588 
315-682-7777 
619-553-4045 
609-866-6253 
206-655-8544 
703-352-1879 
202-767-3882 
703-960-1000 

201- 284-2032 

315-330-3175 

703-347-6445 

415-966-2722 

202- 767-5966 
617-272-3000 
703-883-7033 
202-695-4224 
703-620-7287 
301-394-1961 
505-667-4370 
301-953-5000 
617-377-6879 
215-558-7900 
202-394-3177 
61 9-224-3500 

817- 763-4046 
215-441-2865 
703-64.4-0122 
215-441-1171 
21 5-644-3400 
703-276-6969 

818- 901-2048 
619-939-3651 
617-271-2501 
703-243-2500 

213- 647-9471 

214- 343-7702 
714-896-1565 
703-246-9060 
617-942-2000 
617-942-2000 
303-977-0206 
213-393-0411 
703-876-8003 
301-953-5000 


LAST  NAME 

Bov/e  n 

Bovfflian 

Brain 

Brand stadt 
P rob St 5  Jr. 

B roman 

Brown 

Brown 

Buckley 

Buddenberg 

Buede 

Buemi 

Buick 

Bullen 

Bunn 

Burke 

Burns 

Butz 

Campo 

Cannell 

Carson 

Carson,  Jr. 

Case 

Chan 

Chapin,  Jr. 
Chapman 

Chin 

Chu 

Chu 

Chuang 
Chubb 
Chung 
Cl  aric 
Cl  ark 
Connor 

Converse,  Jr. 

Cook 

Corbeil 

Corbet 

Cronin 

Crowder 

Crowder 

Cummings 

Daly 

Davidson 

Da  vi  s 

Da  vi  s 

DeFoe 

DeRieux 

DeWitt 

Desai 


FIRST  NAME  COMPANY 


PHONE  NUMBER 


Mr.  Robert  D. 
Dr.  Chris  L. 

Mr.  Gary  G. 

Mr.  Jeffrey  C. 
Mr.  Donald  G. 
Mr.  Vincent  P. 
Dr.  Robert  R. 
Mr.  Charles  H. 
Mr.  Peter 
LCDR  Rex  A. 

Dr.  Dennis  M. 
Mr.  Robert  P. 
Mr.  John  R. 

Mr.  George 
Mr.  Roger  A. 

Mr.  H.  Leonard 
Mr.  James  E.  S. 
Mr.  Will  A. 

Mr.  Charles  P. 
Mr.  Marshall  H. 
Mr.  Michael  H. 
Mr.  Robert  R. 
Mr.  Gary  R. 

Mr.  Chi  K. 

CAPT  Robert  W. 
Mr.  Howard  M. 
Dr.  Leonard 
Dr.  Wesley  W. 
Ms.  Cecelia  H. 
Mr.  Ping  C. 

Mr.  Douglas  W. 
Mr.  Charles  W. 
Mr.  Ronald  C. 
Mr.  Terrence  K. 
Mr.  Edward  D. 
Mr.  Leonard  E. 
CW2  Lawrence  R. 
Mr.  Allan  F. 

Mr.  Dan 
Mr.  Terence  M. 
Mr.  Kenneth 
Mr.  Kenneth  V. 
Mr.  C.  Alan 
Mr.  Daniel  A. 
Mr.  John  R. 

Mr.  Edison  M. 
Ms.  Laura  C. 

Mr.  Douglas  N. 
Mr.  Thomas  L. 
Mr.  Ronald  N. 

Dr.  Mukund  N. 


General  Dynamics 

Ball  Systems  Engineering  Div. 

System  Planning  corp. 

GE  Aerospace  Electronics  Sys.  Dept. 

UNISYS  Corporation 

NOSC 

The  Rand  Corporation 
McDonnell  Douglas  Corp. 

E-Systems,  Inc./Melpar  Div. 

US  Coast  Guard  Headquarters 
Bend ix/Deci Sion  Logistics 
Daniel  H.  Wagner,  Assoc. 

Loral  Electronic  Systems 
Booz,  Allen  &  Hamilton,  Inc. 

USAF  ASD/RWZE 

Naval  Air  Systems  Command 

Litton  Data  Systems 

E-Systems 

TASC 

MITRE  Corp 

GTE  Government  Systems 
Nor den  Systems 

Lockheed  Missiles  &  Space  Co.,  Inc. 
M.I.T.  Lincoln  Laboratory 
HQ  COMSPAWARSYSCOM 

USMC  Deputy  Program  Mgr.  Intelligence 

MAGNAVOX/GAC 

W.  W.  Chu  Associates 

NORDA 

NSWC/WO 

US  Army  Center  for  Signals  Warfare 
Rockwell  International 
ITT  Gil  fill  an 

Pacific  Missile  Test  Center 
Corvus  Corporation 
RADC/IRRA 

Joint  Tactical  Fusion  Program 
Technology  Service  Corporation 
TASC 


619-592-5054 

619-457-5550 

703-276-4725 

315-793-7357 

703-620-7846 

619-553-1641 

213-393-0411 

703-247-3514 

703-560-5000 

202-267-1206 

703-860-3678 

21 5-644-3400 

914-964-2828 

703-893-0040 

513-255-995862 

202-692-3965 

818-902-4384 

703-352-0300 

617-942-2000 

617-271-7082 

301-738-8958 

516-845-2498 

703-841-5022 

617-981-7934 

202-692-1095 

703-640-2247 

215-699-8851 

213-454-9314 

601-688-4790 

202- 394-1237 
703-347-6445 
213-420-4910 
818-988-2600 
805-989-9436 
703-893-0228 
315-330-3175 
703-556-2983 

203- 268-1249 
703-558-7458 


U.S.  Army  Cecom  Ctr.  for  Signals  Warfar503-347-6430 
Boeing  Company  206-865-4587 
Gov't,  Info.  Sys/Boeing  Computer  Scienc806-865-4587 


CAS,  Inc 

Litton  Data  Systems 
The  MITRE  Corp. 

NADC 

Naval  Research  Lab. 

Textron  Defense  Systems 
Naval  Surface  Warfare  Center 
Pacific-Sierra  Research  Corp. 
C.  S.  Draper  Laboratory,  Inc. 


205-895-8636 

818-902-4404 

703-883-6041 

215-441-2514 

202-767-2877 

617-381-4766 

703-663-8763 

703-527-4975 

617-258-2224 
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LAST  NAME 

FIRST  NAME 

COMPANY 

PHONE  NUMBER 

Di Palma 

Mr.  Robert  F. 

ESY-Ctr.  for  Advanced  PI anning/Analysis703-352-0300 

Dick 

Dr.  Stephen  G. 

Hazel  tine  Corp. 

516-261-7000 

Dickinson 

Dr.  Philip  C. 

E -Systems/ CAPA 

703-352-0300 

Diggs 

Mr.  James  F. 

Naval  Research  Lab. 

202-767-0489 

Di shman 

Mr.  David  R. 

SAIC 

619-546-6237 

Dool  ey 

Mr.  Richard  P. 

Martin  Marietta  Electronic  Systems 

305-356-6761 

Dotts 

Mr.  Wilfred  R. 

Pacific  Missile  Test  Center 

516-953-3681 

Drillings 

Mr.  Michael 

Army  Research  Institute-BR 

202-274-5572 

Drummond 

Dr.  Oliver  E. 

Hughes  Aircraft  Co. 

213-616-2624 

Dunham 

Lt  Col  Alan  D. 

DARPA 

202-694-5738 

Duniho 

Mr.  Michael  A. 

NSA 

301-688-7608 

Du  shman 

Mr.  Allan 

Dynamics  Research  Corp. 

617-475-9090 

Egan 

Dr.  John  T. 

Naval  Research  Laboratory 

202-767-0762 

Ehl  ers 

Dr.  David  H. 

The  MITRE  Corp. 

206-655-9910 

Ei serman 

Mr.  Gary  K. 

Decision-Science  Applications,  Inc. 

703-243-2500 

Emm er man 

Dr.  Philip  J. 

Harry  Diamond  Labs 

202-394-3000 

Esfandiari 

Dr.  Pashang 

NSWC/WOL 

202-394-1378 

Evans 

Dr.  Robert  H. 

Naval  Research  Laboratory 

202-767-3569 

Everett 

Mr.  James  T. 

Johns  Hopkins  University  APL 

301-792-5000 

Fagarasan 

Dr.  John  T. 

Hughes  Aircraft  Co. 

714-732-0295 

Fa i son 

Mr.  Joseph  C. 

AFWAL/CDJA 

51 3-255-9335 

Falkner,  Jr. 

Mr.  Joe  S. 

COLSA,  Inc. 

205-830-5412 

Feldman 

Ms.  Susan  J. 

Hughes  Aircraft  Co. 

714-732-0545 

Felton 

Mr.  Mark  A. 

Emerson  Electric  Co. 

314-553-2805 

Ferreira 

Mr.  David  M. 

SPAWARS 

202-692-4510 

Figueroa 

Lt  Col  Robert  C 

Intelligence  Community  Staff 

202-376-5560 

Filippelli 

Mr.  Lawrence  J. 

Ball  Systems  Engineering  Div. 

703-528-3337 

Fi orentino 

Mr.  Joseph  S. 

General  Dynamics  Pomona  Division 

714-868-1000 

Fitchek 

Dr.  Joseph  J. 

Naval  Ocean  Systems  Ctr. 

619-553-2486 

Fitz 

Mr.  Michael  J. 

PACMISTESTCTR 

805-989-3706 

FI orence 

Mr.  Gerald  P. 

Unisys  Corporation 

703-620-7226 

Fong 

Mr.  Franklin  M 

Hughes  Aircraft  Co. 

213-647-9464 

Fong 

Mr.  Robert  C. 

TRW  Defense  Systems  Group 

213-217-6249 

Foote 

Dr.  Austin  L. 

Northrop  Advanced  Systems  Di v. 

213-948-6534 

Fowl  er 

Mr.  Thomas  J. 

Wright  Patterson  AFB 

51 3-255-661 1 

Fox 

Mr.  Gregory 

TRW 

703-876-8014 

Freeman 

Dr.  Raymond  F. 

Joint  Tactical  Fusion  PMO  (HQDA) 

703-556-3155 

Freyer 

Mr.  Gustav  J. 

COLSA,  Inc. 

915-779-5899 

Friel 

Ms.  Patricia 

RCA  -  Electronic  Sys.  Div. 

609-722-4257 

Gabriel 

Mr.  John  R. 

Argonne  Nat'l.  Lab. 

312-972-7240 

Gates 

Mr.  Douglas 

The  MITRE  Corporation 

703-883-5948 

Gee 

Dr.  Sherman 

Office  Naval  Technology 

202-696-4791 

Gehman 

Mr.  P.  Robert 

General  Electric/Aerospace 

215-354-1074 

Geist 

Dr.  John  M. 

Harris  Corp.  -  GSS 

305-727-6041 

George 

Mr.  Troy  H. 

UNISYS  Corp. 

703-558-7279 

Gerber 

Mr.  Mark  A. 

ALPHATECH,  Inc. 

617-273-3388 

Gibbons 

Dr.  Gregory  D. 

Systems  Control  Tech. ,  Inc 

415_494-2233 

Gibson 

Mr.  Richard  L. 

Loral  Defense  Systems 

602-925-6548 

Gi  b  son 

Mr.  Richard  L. 

Loral  Defense  Systems 

602-925-6548 

Givens 

LT  Andrew  G. 

US  Coast  Guard 

609-729-8953 

Godfrey 

Mr.  Scott  L. 

FMC  Corporation 

703-663-9291 
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LAST  NAf^E 

Gold 

Goodman 

Goroff 

Grabowski 

Greer 

Grubbs 

Gully 

Hafner 

Hagaman 

Hall 

Hargi s 

Harmon 

Harmon 

Hart 

Hart 

Hart 

Harvey 

Hassett 

Hatleberg 

Haug 

Hayes 

Hearn 

Heath 

Hecht 

Heinz 

Heithecker 

Herting 

Higa 

Hilbert 

Hilga 

Hofmann 

Hofmann 

Hoi  1 iday 

Holt 

Houston- Ludl am 
Hovey,  Jr. 

Hud 

Hughes 

Humphrey 

Hunigan 

Hunt 

Hunter 

Huo 

Jenkins 
Johnson 
Jo  St 
Jung 
Kadar 
Kaisler 
Kampe 
Ka  skowi  tz 


FIRST  NAME  COMPANY 


PHONE  NUMBER 


Mr.  Steven  B. 
Dr.  Irwin  R. 

Mr.  Spencer 
Dr.  Martha  R. 
CDR  Richard  E. 
Ms.  Linda  L. 

Dr.  Sol  W. 

Dr.  Arnold  N. 
Mr.  Harry  T. 

Mr.  David  L. 

Mr.  Gerald 
BG  William  E. 
Mr.  Kenneth  W. 
Mr.  Leo  A. 

Mr.  Thomas  M. 
Mr.  William  A. 
Mr.  Phillip  I. 
Mr.  Daniel  F. 
Dr.  Clancy 
Dr.  Anton  J. 

Dr.  James  C. 

Mr.  Daniel  L. 
Mr.  Oliver  N. 
Dr.  Herbert 
Lt  Gen  Edward  J 
Mr.  Christof  H. 
CAPT  Thomas  L. 
Mr.  Stan  J. 

Mr.  Donald  E. 
Mr.  Stan  J. 

Dr.  Mark  A. 

Mr.  James  B. 

Ms.  Victoria  M. 
Mr.  James  T. 

Mr.  Mark  D. 

Mr.  Herbert  S. 
Ms.  Vicki 
Mr.  Thomas  F. 
Mr.  Clyde  L. 
CAPT  Kirk  A. 

Mr.  Richard  V. 
Ms.  Kathie  A. 
Ms.  Vicki 
Mr.  Karl  B. 

Mr.  Joseph  A. 
CAPT  Randy  J. 
Ms.  Natalie  F. 
Dr.  Ivan 
Mr.  Stephen  H. 
Mr.  Thomas  W. 

Mr.  David  H. 


General  Electric 
Naval  Ocean  Systems  Ctr. 

UNISYS  Corporation 
GE  Tactical  Systems  Dept. 

Space  &  Naval  Warfare  Systems  Command 
Lockheed  Aeronautical  Systems  Co. 
ALPHATECH,  Inc 
Computer  Sciences  Corp. 

HRB-Singer,  Inc. 

HRB-Singer,  Inc. 

Department  of  Defense 

Joint  Tactical  Fusion  PMO  (HQDA) 

Naval  Electronic  Sys.  Eng.  Activity 
The  MITRE  Corporation 
MITRE  Corporation 
National  Security  Agency/ CSS 
Lockheed  Aeronautical  Systems  Co. 


215-354-1262 

619-553-4014 

516-574-3467 

413-494-7606 

202-692-8974 

818-847-1131 

617-273-3388 

619-225-8401 

703-553-6340 

814-238-4311 

301-688-6501 

703-556-2930 

301-862-8250 

617-271-3715 

617-271-8503 

301-688-6511 

818-847-2750 


Martin-Marietta  Aerospace/Naval  Systems301 -682-0027 


Science  Applications  Int'l.  Corp. 
Martin  Marietta  Aero  &  Naval  Systems 
COLSA,  Inc. 

Consultant  -  Hughes  Aircraft  Co. 

US  Army  Aviation  Center 
SoHaR,  Inc. 


619-458-4932 

301-682-1665 

205-830-5412 

714-732-7791 

205-255-3973 

213-935-7039 


Director,  Intelligence  Community  Staff  202-376-5610 
Naval  Air  Development  Center  215-441-3731 
National  Computer  Security  Center  301-859-4450 
Naval  Weapons  Center  619-939-3651 
Analytical  Systems  Engineering  Corp.  703-892-6000 
Naval  Weapons  Center  619-939-3651 
DoD,  US  Army  Human  Engineering  Lab  301-278-5804 
NRL  202-767-3919 
TRW  703-876-8381 
Air  Force  Studies  Analyses  202-695-5387 
Frontier  Technologies,  Inc.  301-266-8244 
US  Army  Center  for  Signals  Warfare  703-347-6594 
BBN  Lab  703-284-4600 
General  Dynamics  817-777-8285 
Westinghouse  Electric  Corp.  301-765-9965 
ASD/VFAC,  WPAFB  513-255-3816 
Pacific  Missile  Test  Center  805-989-9586 
Magnavox  Electronic  Systems  Co.  219-429-6001 
BBN  Lab  703-284-4600 
Communications  Systems  Tech.,  Inc.  301-381-5080 


USMC 

AFWAL/CDJ 

General  Dynamics  Corp. 
Grumman  Aircraft  Sys. 
DARPA 

Norden  Systems 
Tiburon  Systems,  Inc. 


513-255-9335 

714-868-4761 

516-575-5359 

202- 694-1703 

203- 385-5746 
408-371-9400 
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LAST  NAME 

FIRST  NAME 

COMPANY 

PHONE  NUMBER 

Kelly 

Mr.  Robert  J. 

Allied-Signal,  Inc.,  Communications  Di v301 -583-4473 

Kelly 

Mr.  William  T. 

Analytics  Suite  481 

703-359-2850 

Kessler 

Mr.  Keith  J. 

LORAL  Electronic  Systems 

914-968-2500 

Kessler 

Mr.  Otto 

NADC 

215-441-1569 

Kim 

Mr.  Kwang  H. 

The  MITRE  Corp. 

617-271-2501 

King 

Mr.  John  J. 

Unisys  Corp. 

516-574-5423 

Kirshenbaum 

Capt.  Jason 

HQ  USAF  (Air  Staff 

202-695-4561 

Ki ssinger 

Lt  Col  Don  F. 

HQ  USAF/INXXA 

202-695-4561 

Knuff 

Mr.  James  P. 

Logicon 

619-455-1330 

Kola nek 

Dr.  James  C. 

Raytheon  Co. 

805-967-5511 

Krai 

LCDR  Theodore 

SPAWARS 

202-692-3966 

Krueger 

Mr.  Karl  H. 

NSWC 

703-663-8906 

Kuen 

Mr.  Jeffrey  S. 

Pacific  Missile  Test  Center 

805-989-8977 

Kurien 

Dr.  Thomas 

ALPHATECH,  Inc. 

617-273-3388 

Lagana 

Mr.  Richard  A. 

National  Security  Agency 

301-859-4823 

Lamar  (Ret) 

BGen  Kirby 

AFCEA 

703-631-6235 

Lamar  USA  (Ret. ) 

BG  Kirby 

AFCEA 

703-631-6235 

Lampe 

Mr.  Douglas  H. 

TCI 

415-795-5322 

Lancaster 

Dr.  James  E. 

McDonnell  Douglas  Astronautics  Co. 

714-896-4811 

Lane 

Mr.  Patrick  J. 

E-Systems,  Inc. 

703-560-5000 

Lange 

Mr.  David  W. 

DoD/NSA 

301-688-7608 

Lange 

Mr.  David  W. 

National  Security  Agency 

301-688-7608 

Larson,  USAF  (Ret)Maj  Gen  Doyle  E 

Larson  Lectronics,  Inc. 

612-890-9140 

Laskowski 

Dr.  Sharon  J. 

The  MITRE  Corporation 

703-883-7929 

LeVine 

Dr.  Donald  M. 

TRW  Federal  Systems  Group/Cmd.  Sup.  Di v703-968-l 144 

LeVine 

Mr.  Stuart 

Rockwell  International  AMSD 

714-762-8340 

Lee 

Dr.  Yeechun 

Los  Alamos  Nat'l  Laboratory 

505-667-8715 

Lee 

Mr.  Brian  M. 

Raytheon  Co.  (MSD) 

617-274-3211 

Lee 

Mr.  Chai 

Lockheed  Aeronautics  Systems  Company 

805-257-5718 

Lee 

Mr.  J.  Preston 

ORINCON  Corp. 

619-455-5530 

Lehmann 

Mr.  Clark  T. 

McDonnell  Douglas  Astronautics  Co. 

714-896-1863 

Lempicki 

Mr.  Walter  P. 

USAF 

513-255-9958 

Levy 

Mr.  Lawrence  H. 

Naval  Air  Systems  Command 

202-746-2120 

Liggins,  II 

CAPT  Martin  E. 

Rome  Air  Development  Center 

315-330-4431 

Lightstone 

Mr.  Robert  M. 

Boeing  Aerospace  Co. 

215-672-8520 

Lindinger 

Mr.  Joseph  L. 

Naval  Air  Development  Center 

215-441-3189 

Lindsay 

Ms.  Nancy  P. 

E-Systems  Inc.,  Melpar  Div. 

703-385-5880 

Li vingston 

Capt  Mary  E. 

Joint  Tactical  Fusion  Program  Office 

703-556-3173 

LI  inas 

Dr.  James 

Cal  span  Corporation 

716-632-7500 

LoSecco 

Mr.  John  F. 

Synectics  Corporation 

315-337-3510 

Lockie 

Mr.  Doug 

Pacific  Monolifics 

408-732-8000 

Loewenthal 

Dr.  Alex 

Lockheed  Aeronautical  Sys.  Co. 

818-847-6842 

Long 

Mr.  James  P. 

General  Electric 

215-531-1918 

Long 

Mr.  James  P. 

General  Electric  Co. 

215-531-1918 

Lopez,  Jr. 

Mr.  Alfredo 

HQ,  Electronics  Security  Cmd 

512-925-2703 

Loveland 

CAPT  Richard  S. 

Naval  Operational  Intelligence  Center 

202-763-3694 

Luppino 

Mr.  Peter  L. 

OUSD(A) 

202-697-1522 

Maar 

Dr.  James  R. 

National  Security  Agency 

301 -859-6341 

MacMeekin 

Ms.  Nancy  E. 

Naval  Air  Development  Center 

215-441-1720 

Macmeek i n 

Ms.  Nancy  E. 

Naval  Air  Development  Center 

215-441-1720 

Maconachy 

Mr.  Wil liam  V. 

INFOSEC  Awareness 

301-688-8742 
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LAST  NAME 

FIRST  NAME 

COMPANY 

PHONE  NUMBER 

Madan 

Dr.  Rabinder  N. 

Office  of  Naval  Research 

202-696-4217 

Maginn 

Mr.  Mark  T. 

PAR  Government  Systems  Corp. 

315-738-0600 

Mallach 

Mr.  Larry 

Lockheed  Aeronautical  Systems  Co. 

818-847-1405 

Maloney 

Patricia  Sue 

Lockheed,  Advanced  Technology 

512-448-5136 

Mangeri 

Ms.  Lesli  N. 

Pacific-Sierra  Research  Corp. 

703-527-4975 

Mangoubi 

Mr.  Rami  S. 

C.  S.  Draper  Laboratory,  Inc. 

617-258-2262 

Marchette 

Mr.  David  J. 

Naval  Ocean  Systems  Ctr. 

619-553-4049 

Martin 

Mr.  John  W. 

Bendix  Communications  Div. 

301-583-4258 

Masenten 

Dr.  Wesley  K. 

Advanced  Avionics  System  Engineering 

213-948-8979 

Mason 

Mr.  Clark  B. 

Lockheed  Aeronautical  Systems  Co. 

818-847-4134 

Mat sumo to 

Mr.  Paul  M. 

ITT  Gil  fill  an 

818-988-2600 

Mazzuca 

Mr.  Paul  N.  Jr. 

Teleforce  Associates 

301-961-6555 

Mazzuca,  Jr. 

Mr.  Paul 

Tele-Force  Associates 

301-961-6555 

Me C own 

Ms.  Laurie  J. 

TRW 

213-297-3135 

McCoy 

Mr.  Shawn  K. 

Martin-Marietta  Aero  &  Naval  Systems 

301-682-1924 

Me  Da  niel 

Mr.  Eubert  L. 

HQ  ASD/VFEA 

513-255-4921 

McEachern 

Mr.  Robert  H. 

HRB-Singer 

3C1 -459-8655 

McKenney 

Mr.  Terry  L. 

Westinghouse  Electric  Corp. 

301-765-1097 

McLeod 

Mr.  Glenn  A. 

Naval  Surface  Warfare  Ctr. 

301-394-1394 

McNamara 

Mr.  Robert  W. 

PRB  Associates,  Inc. 

301-373-2360 

McNeill 

Dr.  David  L. 

E-Systems 

703-560-5000x279£ 

Mechtel 

Mr.  Gary  H. 

Westinghouse 

301-765-6142 

Meyer 

Mr.  Michael  P. 

Norden  Systems 

516-845-2596 

Mi eras 

Dr.  Harry 

Raytheon  Co. 

617-274-4021 

Mi  1  an 

Mr.  John  M. 

ITT  Gil  fill  an 

818-988-2600 

Milford 

Mr.  John  A. 

E-Systems,  Inc. /Garland  Div. 

214-272-0515 

Miller 

Dr.  Irwin 

Raytheon  Company 

617-860-2856 

Miller 

Mr.  Edwin  H. 

Sanders  Associates 

603-885-4030 

Mi 1 1 er 

Mr.  Robert  C. 

Microwave  Associates,  Inc. 

617-272-5008 

Mills,  Jr. 

Mr.  Harold  W. 

E-Systems,  Inc. 

703-352-0300 

Mil  u ski 

Mr.  John  P. 

PRB  Associates,  Inc. 

301-673-2360 

Mitchell,  Jr. 

Mr.  George  W. 

US  Army  Ctr.  for  Signals  Warfare 

703-347-6791 

Mittenthal 

Dr.  Lothrop 

Teledyne  Electronics 

805-498-3621 

Mi tzel 

Dr.  Glenn  E. 

Johns  Hopkins  Univ./APL 

301-953-5309 

Moe 

Mr.  Gordon  0. 

Pacific-Sierra  Research  Corp. 

703-527-4975 

Moldoff 

Mr.  Barry 

The  MITRE  Corp 

617-271-3466 

Monaghan 

Mr.  Paul  J. 

US  Navy 

513-255-7615 

Moravec 

Mr.  Kipton  S. 

Texas  Instruments 

011498161804753 

Moreno 

Maj  Abel 

USAF 

617-271-4720 

Moroney 

Mr.  Joseph  M. 

RCA  Corp,  Electronic  Sys  Div 

609-722-4256 

Morri s 

Mr.  Lee  G. 

VEDA,  Inc. 

215-672-3200 

Morri  s 

Mr.  William  M. 

Naval  Research  Lab 

202-767-5976 

Mullen 

Mr.  Ronald  E. 

SAIC 

703-749-8709 

Mullens 

Mr.  David  G. 

Boeing  Aerospace 

206-773-3416 

Muller 

Mr.  James  R. 

Synetics  Corporation 

315-337-3510 

Mussmann 

Mr.  David  E. 

Johns  Hopkins  Univ/APL 

301-953-5000 

Mutchler 

Dr.  Carl  N. 

The  MITRE  Corp. 

703-883-5527 

Muzik 

Linda  M. 

INFOSEC  Awareness 

301-688-8744 

Nakamura 

Mr.  Yukio 

Jet  Propulsion  Laboratory 

818-354-7344 

Nell  an s 

Gale  D. 

Geodynamics  Corp 

703-971-9000 

Nel  son 

Mr.  James  B. 

Lockheed  Missile  &  Space  Co.,  Inc. 

512-448-9488 
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LAST  NAME 

FIRST  NAME 

Nel son 

Mr.  Newton  P. 

Ng 

Mr.  Bin  M. 

Nguyen 

Mr.  Huan  H. 

Nichol s 

Mr.  James  R. 

Noble 

Dr.  David  F. 

Noble 

Dr.  Steven  S. 

Nor  seen 

Mr.  John  D. 

Northrup,  IV 

Mr.  Francis  B. 

O'Leary 

Mr.  Bryan  F. 

O'Neill 

Mr.  Patrick  J. 

Oates 

CDR  John  S. 

01  love 

Mrs.  El  izabeth 

Otts 

Mr.  Ralph  H. 

Paasch 

Mr.  John 

Padgett 

Mr.  Philip  J. 

Pari  si 

Mr.  Michael 

Parsons 

Mr.  Ronald  C. 

Pendergast 

Mr.  Stephen  L. 

Perkins 

Mr.  James  A. 

Perras 

CDR  Wayne  I. 

Perry 

Mr.  James  L. 

Pf i ster 

Mr.  Gerhard 

Phibbs 

Mr.  Kenneth  W. 

Pi erami co 

Mr.  Alan  F. 

Pinto 

Mr.  Robert  W. 

PI antenga 

Mr.  Todd  D. 

Powel 

Mr.  Donald  R. 

Pozza,  USMC  (Ret. 

)Colonel  John  B 

Prentiss 

Dr.  Robinson  N 

Presley,  Jr. 

Dr.  Joe  A. 

Presley,  Jr. 

Dr,  Jos.  A. 

Price 

Dr.  Edward  L. 

Price 

Mr.  Charles  F, 

Price 

Mr.  Charles  F. 

Priebe 

Mr.  Carey  E. 

Radcl iffe 

Mr.  Richard  H. 

Rawicz 

Mr.  Harris  C. 

Rawsthorne 

Dr.  Daniel  A. 

Rebel ein 

Mr.  Paul  R. 

Rebovich 

Mr.  George 

Reedy 

Mr.  Robert  W. 

Reil  ly 

Mr.  John  F. 

Reuther 

Mr.  Clifford  S 

Ritchie 

Mr.  Curtis  A. 

Ri tter 

Mr.  Albert  D. 

Ri vera 

Mr.  Jorge  J. 

Robinson 

Ms.  Joyce  M. 

Rock 

Mr.  Carlton  D. 

Rogers 

Mr.  Roland  D. 

Rogers 

Mr.  Steven  A. 

Rolita,  Jr. 

Mr.  Edward  P. 

COMPANY 

Boeing  Aerospace  Co. 

HQ  Electronic  Security  Command 
Naval  Surface  Warfare  Center 
Naval  Weapons  Center 
Engineering  Research  Associates 
Boeing  Military  Airplane  Co. 

Unisys  -  Defense  Systems 
Intelligence  Community  Staff 
General  Dynamics  -  Electronics 
US  Army  Material  Systms  Analysis 
Naval  Intelligence  Automation  Center 
Department  of  Defense 
UNISYS  Corp 
Raytheon  Co. 

McDonnell  Douglas  Astronautics  Co. 

JJM  Systems,  Inc. 

GTE  Government  Systems 
Hughes  Aircraft  Co. 

McDonnell  Dougl as  Corporation 

CINCPACFLT 

SENSIS  Corporation 

ITT  Gilfillan 

Westinghouse  D  &  E  Center 

Technology  Service  Corp. 

TASC 

Magnavox  Electronic  Systems  Co 
McDonnell  Douglas  Astronautics  Co. 
AFCEA 

Science  Applications  Int'l.  Corp. 
ORINCON  Corp. 

ORINCON  Corporation 
FMC  Corp. 

TASC 

TASC 

NOSC 

Joint  Tact.  Fusion  Prg.  Mgt.  Office 
Lockheed  Electronics  Co.,  Inc. 

BDM 

Honeywell  Inc.  Systems  &  Research  Ctr. 
The  MITRE  Corporation 
Hughes  Aircraft  Co. 

PAR  Government  Systems  Corp. 

TRW 

E -Systems 
NAVSEASYSCOM 

The  Johns  Hopkins  University 
Interstate  Electronic  Co. 

UNISYS,  Defense  Systems 
Interstate  Electronics  Corp. 

Lockheed  Austin  Division 
Rolita  Consultants 


PHONE  NUMBER  ■ 

206-655-8544 

51 2-925-1  60T; 

202-394-1456 

619-939-2335 

703-734-8800  ■ 

206-241-3426 

703-620-7860 

202- 376-5568  ; 
619-573-7718 
301-278-6429 
301-763-3514 
301-688-7162 
612-681-6207 
617-274-2748:: 
703-276-4618 
215-672-3660 
301-294-8627  ; 
714-732-2579 
703-247-3513 
808-471-0759  . 
315-682-7777  : 
818-988-2600 
301-379-1008 . 

203- 268-1249 
617-942-2000 
219-429-6707-- 
714-896-1493 
703-631-6238 
714-640-8662,, 
619-455-5530 
619-455-5530 
703-663-9291  : 
617-942-2000 
617-942-2000 
61 9-553-4048  ; 
703-556-2952 

201- 757-1600 
703-848-6338  . 
612-782-7059 ' 
617-271-8261 

213- 334-4138  - 
315-738-0660 
703-876-4153 

214- 272-0515  ; 

202- 692-9768 

714-758-4175  - 

612-681-6252 

714-758-4072 

512-448-5236 

703-791-5505 


LAST  NAME 

FIRST  NAME 

COMPANY 

PHONE  NUMBER 

Romero 

Mr.  Joseph 

GTE  Government  Systems 

415-966-3156 

Romi  e 

Mr.  Roger  E. 

AFWAL/AAWD-1 

513-255-6648 

Roney 

Mr.  Philip  M. 

General  Electric  Co  (MC3I) 

215-354-5110 

Rosen 

Dr.  Julie  A. 

SAIC 

703-556-7354 

Rothman 

Mr.  Richard  G. 

Navy  Dept.  SPAWAR 

202-692-8921 

Roth  rock 

Col  John 

USAF/INYX 

202-695-9066 

Ruland 

Mr.  Timothy  P. 

Westinghouse  Defense  Center 

301-379-1005 

Russo 

LCDR  Mary  S. 

Defense  Comm.  Agency/ Advanced  Tech  Off 

703-437-2506 

Sail sbury 

Mr.  Robert  A. 

Tele-Force  Associates,  Inc. 

301-961-6555 

Samaan 

Dr.  Jacob  E. 

IBM  Federal  Systems  Di vision 

703-841-7300 

Sando 

2nd  LT  Kermit  A 

USAF 

513-255-7615 

Santa 

Mr.  Joseph  E. 

TRW  Defense  Systems  6rou[ 

213-217-3081 

Schlegel 

Mr.  Paul  T. 

Naval  Air  Systems  Command 

202-692-2511 

Schneberger 

CDR  Scott  L. 

Navy-Chief  of  Naval  Operations 

202-694-0297 

Schneider 

Mr.  Raymond  J. 

Uni  sys 

703-528-3902 

Schweiter 

Dr.  Gail  Ann 

Daniel  H.  Wagner,  Assoc. 

215-644-3400 

Scott 

Mr.  Godfrey  L. 

FMC  Corp. 

703-663-9291 

Scott 

Mr.  Godfrey  L. 

FMC  Corporation 

703-663-9291 

Scott 

Mr.  Peter  A. 

Ford  Aerospace  &  Communications  Corp. 

714-720-6752 

Scul ly 

Dr.  John  W. 

U.  S.  Army  Harry  Diamond  Labs 

202-394-2300 

Seal  s 

Mr.  J.  Dennis 

AT&T  Bell  Laboratories 

201-386-3373 

Selzer 

Mr.  Fred 

Ford  Aerospace  Corp. 

714-720-6283 

Shade 

Mr.  Robert  A. 

Hazel  tine  Corp. 

516-261-7000 

Shankland 

Dr.  Donn  G. 

Boeing  Computer  Services 

206-865-3520 

Sherman 

Mr.  Frederick  W 

General  Dynamics/ Valley  Sys.  Div. 

714-945-8460 

Shirazi 

Mr.  Mehdi 

AF  Wright  Aeronautical  Labs. 

513-255-5987 

Shoenfeld 

Dr.  Peter 

SAIC 

703-821-4466 

Silagyi 

Mr.  Ernest  S. 

Raytheon  Co. 

617-271-1040 

Silbert 

Mr;  Mark  E. 

Naval  Air  Development  Center 

215-441-2556 

Silver berg 

Ms.  Marjorie  H 

General  Electric  Co. 

215-354-3179 

Si Ivey 

Mr.  Paul  E. 

The  MITRE  Corp. 

617-271-3497 

Simmen 

Mr.  Robert  L. 

Radar  Control  Systems  (RADCON) 

415-484-4066 

Simpson 

Mr.  John  W. 

A1 1 ied-Signal 

301-583-4499 

Singer 

Mr.  Nicholas  C. 

E-Systems  CAPA 

703-352-0300 

Sipos 

Mr.  Jon  C. 

A1 1 i ed-Si gnal  I nc . 

301-583-4112 

Smi  th 

Mr.  Richard  H. 

Ford  Aerospace  Corp.  ' 

415-852-4784 

Smith 

Mr.  W.  Terry 

GTE  Government  Systems 

415-694-1823 

Solomon 

Mr.  Sterling  I. 

The  Analytic  Sciences  Corporation 

703-558-7400 

Somoano 

Mr.  Robert  B. 

Jet  Propulsion  Laboratory 

818-354-2213 

Stamberger 

Mr.  Frederick  J 

Hughes  Aircraft  Company 

714-732-1210 

Stapl eton 

Mr.  Ronnie  A. 

Naval  Surface  Warfare  Center 

703-663-8906 

Staton 

Mr.  Robin  R. 

Naval  Surface  Warfare  Ctr. 

703-663-7911 

Steinberg 

Mr.  Alan  N. 

The  Analytic  Sciences  Corp. 

704-734-4100 

Steinberg 

Mr.  Richard  A. 

Johns  Hopkins  University 

Stein hacker 

Mr.  Mark 

Unisys  Corp. 

516-574-3317 

Steinley 

Mr.  Michael  M. 

Naval  Air  Development  Center 

215-441-7038 

Stephen 

Dr.  Dick  G. 

Hazel  tine  Corp. 

516-261-7000 

Stepp 

Mr.  William  E. 

Lockheed  Aeronautical  Systems  Co. 

818-847-5562 

Stigl itz 

Dr.  Irvin  G. 

MIT  Lincoln  Laboratory 

617-981-7440 

Stoltz 

Mr.  John  R. 

Advanced  Systems  Concepts,  Inc. 

503-386-2225 

Storck 

Mr.  Clarence  E. 

The  MITRE  Corp. 

619-223-2307 
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PHONE  NUMBER 


LAST  NAME 

Studeman 

Sullivan 

Sutherlln 

Suycott 

Ta  sker 

Teal 

Terzian 

Thompson 

Thomsen 

Tolbert 

Toman 

Torre 

Travi s 

Trenchard 

Tuttle 

Van  Doren 

Vanden  Dries 

Vannicola 

Ventrigl 1o 

Vogt 

Vu 

Waagen 

Wallace 

Waller 

Waltz 

Wanless 

Wenocur 

Werner 

Wesley 

White 

Whitlow 

Whittenberger 

Widdoss 

Wiener 

Wiener,  II 

William 

Williams 

Williams 

W11  son 

Windsor 

Wing 

Winkler 

Wishner 

Wiss 

Wofford 

Wohl 

Wolf 

Wolff 

Womack 

Woods 

Workman 

Wright 


FIRST  NAME  COMPANY 


RADM  William  0. 
Mr.  John  F. 

LCDR  Charles  T. 
LT  Mark  L. 

Mr.  Taymond 
Mr.  James  A. 

Mr.  Richard  C. 
Mr.  James  P. 
CAPT  Kurt  E. 

Mr.  David  C. 

Mr.  Donald  J. 
Mr.  Frank 
Mr.  Tim  E. 

Dr.  Herbert  A. 
Dr.  Paul  G. 

Mr.  Glennon  L. 
Mr.  Paul  E. 

Dr.  Vincent  C. 
Mr.  Frank  J. 

Mr.  Gregory  S. 
Mr.  Cuong  M. 

Mr.  Don  E. 

Dr.  Timothy  P. 
Mr.  John  S. 

Mr.  Edward  L. 
Mr.  Loren  E. 

Dr.  Roberta  D. 
Mr.  Keith  M. 

Ms.  Debora  M. 
Mr.  Franklin  E. 
Mr.  Bert  L. 

COL  Steven  J. 
Mr.  Monte  G. 

Dr.  Howard  L. 
Dr.  Daniel  F. 
Mr.  Kelly  T. 

Mr.  Carl  E. 

Mr.  Elmer  F. 

Dr.  Charles  J. 
Mr.  Stephen  W. 
Dr.  David  H. 

Mr.  Gary  L. 

Dr.  Richard  P. 
Mr.  Victor  R. 
Mr.  Floyd  C. 

Dr.  Joseph  G. 
Mr.  R1 chard  G. 
Mr.  Thomas  0. 
Mr.  Rodney  A. 
Mr.  Edward  C. 
Mr.  Billy  J. 

Mr.  David  L. 


Director  of  Naval  Intelligence 
The  MITRE  Corp. 

Space  4  Naval  Warfare  Systems  Cmd. 
Pacific  Missile  Test  Center 
Tasker  Assoc. 

Harris  Corporation 

TRW  Defense  Systems  Group 

Eaton  Corporatlon/AIL  Div. 

Rome  Air  Development  Center 
Martin  Marietta  Aero  4  Naval  Systems 
Loral  Electronic  Systems 
Hazel  tine  Corp. 

TRW 

Westinghouse  Electric  Corp. 

TRW 

National  Security  Agency 
USAF 

Rome  Air  Development  Center 
Naval  Sea  Systems  Comand 
Hughes  Aircraft  Co. 

Naval  Surface  Warfare  Center 
Naval  Ocean  Systems  Center 
M.I.T.  Lincoln  Laboratory 
HQ  TRADOC 

Allied-Signal  Inc. 

Lockheed  Missiles  4  Space  Co. 

Daniel  H.  Wagner  Assoc. ,  Inc. 
GE/RCA-Electronic  Systems  Department 
Westinghouse  Electric  Corp. 

Naval  Ocean  Systems  Center 
ASD/AEIE  Navy 
HQ  USAF/XOOR 
Surface  Warfare  Systems 
Office,  Chief  of  Naval  Oper. 
University  of  Virginia 
Analytics 

Martin  Marietta  Info.  4  Comm.  Sys. 
Naval  Research  Lab 
Penn  State  University 
TRACOR  Aerospace,,  Inc. 

Naval  Research  Lab. 

Analytics 

Advanced  Decision  Systems 
Naval  Surface  Warfare  Center 
US  Army  Ballistic  Research  lab. 
ALPHATECH,  Inc. 

Westinghouse  Defense  Center 
Uni  sys 

GE/RCA  Electronic  Systems  Division 
Westinghouse  Electric 
The  MITRE  Corp. 

Naval  Sea  Systems  Command 


202-695-3944 
617-271-5624 
202-692-9113 
805-989-8977 
213-472-3869 
305-984-6448 
213-217-4799 
516-385-2335 
315-330-4049 
301-682-3046 
914-964-3878 
516-261-7000 
213-297-3132 
301-765-7503 
703-876-8751 
301 -859-4827 
513-255-9956 
315-330-4437 
202-692-9538 
213-334-2598 
202-394-4116 
619-553-4046 
617-981-2848 
804-727-3273 
301-583-4245 

512- 448-5774 
215-644-3400 
609-866-7704 
301-765-6142 
619-553-4036 

513- 255-7615 
202-697-2475 
612-687-1707 
202-694-4770 
804-924-7542 
703-359-2850 
303-977-0958 
202-767-3569 
814-863-4102 
512-926-2800 
202-767-9829 
703-359-2850 
415-960-7300 
703-663-8906 
301-278-6657 
617-273-3388 
301-379-1023 
612-681-6788 
609-722-7993 
301-765-2463 
512-675-9640 
202-692-3796 
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LAST  NAME 

FIRST  NAME 

COMPANY 

PHONE  NUMBER 

Wu 

Mr.  Kepi 

DoD,  Strategic  Defense  Initiative 

202-693-1530 

Wuenschel 

Mr.  Edward  F. 

GTE  Government  Systems 

415-966-3887 

Yanek 

Mr.  Stephen  P. 

Johns  Hopkins  University 

301-953-6316 

Yannone 

Mr.  Ronald  M. 

General  Electric  (INEWS  JVT) 

603-885-3736 

Youngberg 

Mr.  Dean  A. 

HQ  LABCOM 

301-677-3376 

Zabri skie 

Mr.  George  R. 

CAS,  Inc. 

205-895-8600 

Zak 

Mr.  John  W. 

TRW 

Zentner 

Mr.  James  R. 

USAF/F-15  SPO 

513-255-4921 

Zi  timerman 

Mr.  Jeffrey  M. 

Technology  for  Communication  Int'l. 

301-796-7300 
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